- ExperienceOne Golang APIKit
The APIKit enables the rapid development of Web APIs with Golang by providing the functionality to (re-)generate the communication layer of the API based on an OpenAPIv2 (Swagger) definition, both for the client and server part. It also helps with one-time generation of stubs for the server-side endpoint handlers. The generated API code does fully handle (and thus hide) the HTTP layer of the application, so the developer can focus on business logic implementation. Integration between API core and HTTP layer is handled via generated structs and interfaces. The API HTTP layer can be regenerated from the OpenAPIv2 definition without breaking the integration with the API core code. This enables a "definition first" approach that ensures a 100% match of OpenAPI / Swagger definition and the implemented API.
Specifically the APIKit contains a CLI to generate and update the following API related items:
- standard project directory structures and common files
- validation of OpenAPIv2 (Swagger) definition
- HTTP API client based on an OpenAPIv2 (Swagger) definition
- HTTP API server based on an OpenAPIv2 (Swagger) definition
- stubs for server API endpoint handlers
See the compliance list for a detailed list of supported OpenAPIv2 features.
- Golang 1.11 or higher
- make (if you want to use the Makefile)
git clone [email protected]:ExperienceOne/apikit.git
cd apikit
make installThe command apikit project <dest.dir> <path/of/package> generates a standard project directory.
cd go/src
$GOPATH/bin/apikit project myproject myprojectThe project command takes the name of the directory to create and the path of the main package as arguments It creates a CLI stub in the cmd subdirectory and a .gitignore file.
Note that it is not necessary to use the standard project structure with the APIKit.
The next step is to define your API. It can be useful to use a Swagger editor for this task. By convention the Swagger definition is stored in the doc subdirectory.
For showcasing the following example definition will be used:
---
swagger: "2.0"
info:
title: "vis-admin"
version: "1.0.0"
consumes:
- "application/json"
produces:
- "application/json"
paths:
/api/client/{clientId}:
put:
summary: "Create or update client"
operationId: "CreateOrUpdateClient"
consumes: []
parameters:
- name: "X-Auth"
in: "header"
required: true
type: "string"
description: "Authentication token"
- name: "clientId"
in: "path"
required: true
type: "string"
description: "Client ID"
- name: "body"
in: "body"
required: true
schema:
$ref: "#/definitions/Client"
responses:
200:
description: "Updated"
schema:
$ref: "#/definitions/Client"
201:
description: "Created"
schema:
$ref: "#/definitions/Client"
400:
description: "Malformed request body"
403:
description: "Not authenticated"
405:
description: "Not allowed"
definitions:
Client:
type: "object"
required:
- "id"
- "name"
properties:
id:
type: "string"
name:
type: "string"
activePresets:
type: "string"The command apikit validate <api.yaml> validates against the given OpenAPIv2 / Swagger definition.
cd myproject
$GOPATH/bin/apikit validate doc/myproject.yamlThe apikit generate <api.yaml> <dest.dir> <package> command generates the client and server API based on an OpenAPIv2 definition. Use --only-client if the server component and --only-server if the client component is not needed.
$GOPATH/bin/apikit generate doc/myproject.yaml api apiNote that all endpoints need to have an operation ID to generate successfully. The generated code should not be edited manually. Instead, the OpenAPI definition should be updated and the source regenerated with APIKit generate command.
The client.go file contains an interface defining the programming interface of the API.
type VisAdminClient interface {
CreateOrUpdateClient(request *CreateOrUpdateClientRequest) (CreateOrUpdateClientResponse, error)
}The request parameter contains all data that is sent to the server with the API call. Every API call returns a response interface that can be casted to the actual, HTTP return code specific struct, and an error code if the communication with the server went wrong. The content of the response is only valid if the error return value is nil.
The client.go file also contains an implementation of the programming interface using HTTP calls to the actual API. There is a constructor for the client implementation which takes an http.Client and the API base URL as parameters. Via the http.Client network handling and additional http.RoundTripper can be configured and integrated with the API client.
The steps for using the client to make an API call are:
- create the client implementation via constructor
- fill request struct with appropriate data
- call the API via the function for the endpoint
- check error return value
- check HTTP status code of the response
- cast response to correct struct
This is illustrated in the following example:
package myproject_test
import (
"myproject/api"
"net/http"
"testing"
)
const (
baseUrl = "http://api.myproject.com/"
testClientId = "clientId"
testApiKey = "apiKey"
testName = "name"
)
func TestClient(t *testing.T) {
// create instance of API client
client := api.NewVisAdminClient(&http.Client{}, baseUrl, api.Opts{})
// create request struct
request := api.CreateOrUpdateClientRequest{
ClientId: testClientId,
XAuth: testApiKey,
Body: api.Client{
Id: testClientId,
Name: testName,
},
}
// call API with request
response, err := client.CreateOrUpdateClient(&request)
if err != nil {
t.Fatal(err)
}
// check HTTP return code
if response.StatusCode() != http.StatusOK {
t.Fatalf("API returned error code %d", response.StatusCode())
}
// cast response appropriatly and use returned information
t.Log(response.(*api.CreateOrUpdateClient200Response).Body.Name)
}Note that for testing the API can be mocked by a local implementation of the API client programming interface.
The file server.go contains an a full HTTP server that serves the specified API. Additional to the defined endpoints it contains the /spec endpoint that delivers the OpenAPIv2 / Swagger definition the server was generated with. The /spec endpoint can be used to visualize the API via UIs like Swagger UI.
The implementation of the API endpoints has to be done in handler functions that take the request data as parameter and return an implementation of the endpoint-specific response interface. The handler functions need to be registered before starting the server. If there is no handler registered for an endpoint, the server will respond with status code 404 as default.
The steps for using the API server are:
- create the server via constructor
- register endpoint handlers
- start listening on network socket
The following example shows how the server is initialized and than called by the generated client (see above).
package myproject_test
import (
"context"
"myproject/api"
"net/http"
"testing"
"time"
)
const (
baseUrl = "http://localhost:8080"
testClientId = "clientId"
testApiKey = "apiKey"
testName = "name"
)
func CreateOrUpdateClient(ctx context.Context, request *api.CreateOrUpdateClientRequest) api.CreateOrUpdateClientResponse {
return &api.CreateOrUpdateClient200Response{
Body: request.Body,
}
}
func TestServer(t *testing.T) {
// create server instance
server := api.NewVisAdminServer(nil)
// register endpoint handler
server.SetCreateOrUpdateClientHandler(CreateOrUpdateClient)
// start server to listen for requests
go func() {
t.Log(server.Start(8080))
}()
time.Sleep(1 * time.Second)
// create instance of API client
client := api.NewVisAdminClient(&http.Client{}, baseUrl, api.Opts{})
// create request struct
request := api.CreateOrUpdateClientRequest{
ClientId: testClientId,
XAuth: testApiKey,
Body: api.Client{
Id: testClientId,
Name: testName,
},
}
// call API with request
response, err := client.CreateOrUpdateClient(&request)
if err != nil {
t.Fatal(err)
}
// check HTTP return code
if response.StatusCode() != http.StatusOK {
t.Fatalf("API returned error code %d", response.StatusCode())
}
// cast response appropriate and use returned information
t.Log(response.(*api.CreateOrUpdateClient200Response).Body.Name)
// stop server
server.Stop()
}For one-time generation of stubs for the endpoint handlers the command apikit handlers <api.yaml> <dest.go> <package> <api/package/path> can be used. Especially for large APIs it saves some typing work by writing out the boilerplate code for the handlers.
$GOPATH/bin/apikit handlers doc/myproject.yaml handlers.go myproject myproject/apiFor larger projects grouping the endpoint handlers in service classes is convenient. A service stub can be generated with apikit service <api.yaml> <dest.go> <package> <tag> <api/package/path>. The command will generate the endpoints that are tagged with tag only.
$GOPATH/bin/apikit service doc/myproject.yaml client_service.go myproject client myproject/apiThe generated server does validate the request against the constraints defined in the OpenAPIv2 specification. If the validation fails, the server will respond with a 400 Bad Request status code to the client.
To pass more information about the invalid data to the client, the 400 Bad Request response with the body type ValidationErrors has to be added to the endpoint. The generated server will automatically use it to send details about malformed data to the client.
# ... endpoint definition ...
responses:
400:
description: "Bad request"
schema:
$ref: "#/definitions/ValidationErrors"
definitions:
ValidationErrors:
type: "object"
properties:
Message:
type: "string"
Errors:
type: "array"
items:
$ref: "#/definitions/ValidationError"
ValidationError:
type: "object"
properties:
Message:
type: "string"
Field:
type: "string"
Code:
type: "string"
The generated types do reflect if a field is required or not by making not required fields pointers. If a non-required field is not present in the request data or a field is present, but has the JSON value null, the pointer will be set to nil. Otherwise the pointer will point to the actual data. Required fields are not allowed to be not present or to be null. If they are, a 400 Bad Request response is returned to the client.
The proper handling of required and non-required fields allows to correctly distinguish between zero values ("", 0, false) and non-present values. Note that the default status for fields in OpenAPIv2 is non-required. To avoid nil pointer checks mark as many fields required as appropriate.
The string validation supports the minLength, maxLength and the pattern attributes. pattern allows to specify a regular expression that the string has to match. If the string has the format uuid, url or email, it is automatically matched against the corresponding regular expression.
The integer validation supports the minimum, maximum, exclusiveMinimum and exclusiveMaximum attributes.
The client needs to send the correct content type header matching the consumes attribute in the OpenAPIv2 definition. If not, the server responds with a 415 Unsupported media type error.
The APIKit generator has limited enum support. It is required that every enum definition occurs only once in the whole OpenAPIv2 specification. The best way to achieve this is to reference enum types where they are used and have a global definition of the enum type in the definitions section of the specification.
Car:
type: object
required:
- driveConcept
properties:
driveConcept:
$ref: '#/definitions/DriveConcept'DriveConcept:
type: string
enum:
- COMBUSTOR
- HYBRID
- ELECTRIC
- FUELCELL
- UNDEFINEDThis produces the following code in types.go:
type DriveConcept string
const (
DriveConceptCOMBUSTOR DriveConcept = "COMBUSTOR"
DriveConceptHYBRID DriveConcept = "HYBRID"
DriveConceptELECTRIC DriveConcept = "ELECTRIC"
DriveConceptFUELCELL DriveConcept = "FUELCELL"
DriveConceptUNDEFINED DriveConcept = "UNDEFINED"
)
type (
Car struct {
DriveConcept DriveConcept `json:"driveConcept" bson:"driveConcept"`
}
)An error logger can be passed to the server via the ErrorHandler attribute of the ServerOpts struct. The logger will be called when something unexpected goes wrong inside the HTTP layer.
The ServerOpts struct can be used to pass a function to the generated server that is executed during server startup and has access to the internal router object. It is defined as func(router *routing.Router).
With this function routes that are not defined in the OpenAPIv2 specification or are implemented by packages that come with an own HTTP handler can be added to the server.
import(
...
"github.com/go-ozzo/ozzo-routing"
)
func NewServer() {
onStart := func(router *routing.Router) {
router.Any("/metrics", func(ctx *routing.Context) error {
metrics.ServeHTTP(ctx.Response, ctx.Request)
return nil
})
}
server := api.NewVisAdminServer(&api.ServerOpts{OnStart: onStart})Functions that shall be executed every time an endpoint is called can be added to the server and to individual handlers via middleware components. A middleware is defined by a struct that holds the handler function (defined in ozzo-routing) and a flag, if the middleware shall be executed before or after the endpoint handler.
Middleware struct {
Handler routing.Handler
After bool
}Server-wide middleware components can be passed via the ServerOpts struct.
middleware := []api.Middleware{{Handler: middleware.RequestID().Handler}}
server := api.NewVisAdminServer(&api.ServerOpts{Middleware: middleware})Handler-specific middleware can be added via a var-arg list at handler registration.
sessionMiddleware := xcustomer.Middleware{Handler: middleware.SessionHandler(s.sessions)}
server.SetCreateOrUpdateClientHandler(CreateOrUpdateClient, sessionMiddleware)Middleware can use c.Request.Context() object to pass data to the handler function. For example the session handler can check for a valid session token and load session data into the context object of the handler if successful.
func SessionHandler(service session.Service) routing.Handler {
return func(c *routing.Context) error {
authToken := c.Request.Header.Get(XAuthTokenHeader)
if len(authToken) == 0 {
return xcustomer.NewHTTPStatusCodeError(http.StatusBadRequest)
}
sess, err := service.GetSession(authToken)
if err != nil {
if err.StatusCode == http.StatusInternalServerError {
return xcustomer.NewHTTPStatusCodeError(http.StatusInternalServerError)
}
return xcustomer.NewHTTPStatusCodeError(http.StatusUnauthorized)
}
if sess == nil {
return xcustomer.NewHTTPStatusCodeError(http.StatusUnauthorized)
}
// add session data to the context of the endpoint handler
ctx := context.WithValue(c.Request.Context(), xcontext.SessionKey, sess)
c.Request = c.Request.WithContext(ctx)
return c.Next()
}
}The APIKit middleware package provides a component for server-side request and response logging. The middleware records request and response and forwards it to a user-defined log function before sending the data to the client. LogResponseWriter is part of the APIKit middleware package.
rrLogger := func(r *http.Request, w *LogResponseWriter, elapsed time.Duration) {
// ... print request and response infos as suitable
}
middleware := []api.Middleware{{Handler: middleware.Log(rrLogger)}}There is also a convenience wrapper for the log function.
timeFormat := time.RFC3339
pathsToIgnore := []string{...} // endpoints not to log
logger := logFunc func(entity middleware.LogEntry, values ...interface{}) {
// use entity object to print log
}
rrLogger := middleware.NewLogger(timeFormat, pathsToIgnore, logger)In order to comply with the GDPR no data that could identify a person must be stored in log files. Therefore, a middleware logger is provided that can mask specified fields in JSON request and response bodies.
fieldsToMask := []string{...} // list of JSON paths
typesToMask := []reflect.Type{...} // list of types
rrLogger := middleware.NewMaskingLogger(timeFormat, fieldsToMask, typesToMask, pathsToIgnore, logger)The APIKit roundtripper package contains components for client-side request and response logging. The simplest roundtripper forwards the HTTP request and response to a log function. The roundtripper.Use() function wraps the transport of an HTTP client with the given RoundTripper.
logFunc := func(req *http.Request, resp *http.Response) {
// ... log request and response content ...
}
client := roundtripper.Use(&http.Client{}, roundtripper.Logger(logFunc))Additionally there is a convenience roundtripper that does an httputil.Dump..() on request and response and forwards the result a generic logging function.
logger := func(a ...interface{}) {
// log a[0]
}
client := roundtripper.Use(&http.Client{}, roundtripper.Dump(logger))The logging function is defined as an interface var-arg list so that it's possible to use standard functions of logging frameworks like logrus. The roundtripper does always call it with one string parameter.
Up- and downloading binary data in JSON format requires to put whole files in memory while marshalling / unmarshalling the JSON. This can quickly overwhelm the server. A better approach is to handle files as streams. The APIKit supports this via the type: file attribute.
When uploading a file the content type multipart/form-data (consumes attribute) is required.
/upload:
post:
summary: Upload a file with others data
consumes:
- multipart/form-data
operationId: PostUpload
parameters:
- in: formData
name: upfile
type: file
description: the file to upload
- in: formData
name: note
type: string
maxLength: 4000
pattern: "^[0-9a-zA-Z ]*$"
description: Description of file
responses:
'200':
description: Status 200
'500':
description: Status 500The request struct that is passed to the endpoint handler now contains a FormData object that includes an io.ReadCloser to access the file content. Note that the io.ReadCloser needs to be closed by the handler to prevent memory leaks.
func PostUpload(ctx context.Context, request *PostUploadRequest) PostUploadResponse {
defer request.FormData.Upfile.Content.Close()
data, err := ioutil.ReadAll(request.FormData.Upfile.Content)
if err != nil {
log.Println(fmt.Sprintf("error reading uploaded file (%v)", err))
return new(PostUpload500Response)
}
// .. do something with data
return new(PostUpload200Response)
}To deliver a file as stream to the client, the response body must have the type: file. In the example below the Content-Type header specified explicitly to overwrite the Content-Type generated by the produces attribute.
'/download/{image}':
get:
produces:
- image/png
summary: Retrieve a image
description: Retrieve a image
operationId: DownloadImage
parameters:
- in: path
name: image
description: The image name of the image
type: string
required: true
responses:
'200':
description: image to download
schema:
type: file
headers:
Content-Type:
type: "string"
'500':
description: Internal server errorThe handle than has to pass an io.ReadCloser with the response. The open stream is closed in the HTTP layer of the generated server.
func DownloadImage(ctx context.Context, request *DownloadImageRequest) DownloadImageResponse {
file, err := os.Open(request.Name)
if err != nil {
return &DownloadImage500Response{}
}
return &DownloadImage200Response{Body: file, ContentType: "image/png"}
}If you have found a bug or would like contributed to the project please check out our Contribution Guidelines.
After making changes to the APIKit source do run the testsuite via the Makefile.
make testWhen there were changes to the internal/framework package, update the framework files before committing and building the executable.
make frameworkWhen debugging code generation consider enabling the debug mode of the generate command. When activated the generator will save the generated code regardless of errors into a temporary file and it print its location to the terminal.