Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[azopenaiassistants] First checkin for Azure OpenAI assistants #22369

Merged
merged 29 commits into from
Mar 5, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
29 commits
Select commit Hold shift + click to select a range
668c6a2
A package for OpenAI assistants.
Jan 24, 2024
1348d5b
let's try this
richardpark-msft Feb 10, 2024
56990d3
Fixing recordings.
Feb 10, 2024
2b0cb06
New recordings.
Feb 10, 2024
1fba95b
Bring back another test, start getting our CI elements in place.
Feb 10, 2024
cb28116
Updating conversation a bit.
Feb 10, 2024
cba3e81
Accidentally modified a test.
Feb 10, 2024
5b1e478
Making some updates to get the surface into usable shape:
Feb 16, 2024
aaf80bf
New test, new recording.
Feb 16, 2024
d1374c1
Normalize newlines for our tests.
Feb 16, 2024
2d61987
adding in required files
Feb 16, 2024
a6ea25e
- CONTRIBUTING.MD is personalized per package, so just fixing some st…
Feb 16, 2024
2628251
Temporarily use a link to something that does exist so I can get past…
Feb 16, 2024
3d85ea9
Fixing more links, centralizing into the bottom set of links.
Feb 16, 2024
7045a04
- Regenerated, which made the input parameter an io.ReadSeeker for up…
Feb 17, 2024
d1e0c7e
Docs
Feb 17, 2024
a981da6
Fix missing docs.
Feb 17, 2024
89b7bbc
Temporarily disable live testing.
Feb 20, 2024
7775edc
Bring it back!
Feb 20, 2024
d97469d
- Turn on live tests again now that I have the keyvault variables lin…
Feb 21, 2024
c1fcd18
- Accomodating a non-deterministic test. There is a recording that ca…
Feb 22, 2024
2984b42
Fixing test, the expected filename is different with the later versio…
Feb 27, 2024
c0a4ad7
Use the 'Unreleased' keyword instead.
Feb 27, 2024
75a7cf1
Disabling a test until we get all the perms worked out.
Feb 27, 2024
9bb3c84
Rerecord.
Feb 28, 2024
2edc936
Making `clientData` a field instead of an anonymous embed.
Mar 4, 2024
11cd906
Updating readme to have a better intro sentence.
Mar 4, 2024
4edc5dc
Add release date, hopefully release tomorrow provided there are no ne…
Mar 4, 2024
3cbb925
Updating to the merge commit on main (no change)
Mar 4, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions sdk/ai/azopenaiassistants/CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
# Release History

## 0.1.0 (2024-03-05)

* Initial release of the `azopenaiassistants` library
115 changes: 115 additions & 0 deletions sdk/ai/azopenaiassistants/CONTRIBUTING.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,115 @@
# Contributing Guide

> NOTE: these instructions are for fixing or adding features to the `azopenaiassistants` module. To use the module refer to the readme for this package: [readme.md](https://github.com/Azure/azure-sdk-for-go/tree/main/sdk/ai/azopenai/README.md).

This is a contributing guide for the `azopenaiassistants` package. For general contributing guidelines refer to [CONTRIBUTING.md](https://github.com/Azure/azure-sdk-for-go/blob/main/CONTRIBUTING.md).

The `azopenaiassistants` package can be used with either Azure OpenAI or OpenAI's public service. New features are added using our code generation process, specified using TypeSpec [TypeSpec](https://github.com/Microsoft/typespec), which details all the models and protocol methods for using OpenAI.

### Prerequisites

For code fixes that do not require code generation:
- Go 1.18 (or greater)

For code generation:
- [NodeJS (use the latest LTS)](https://nodejs.org)
- [TypeSpec compiler](https://github.com/Microsoft/typespec#getting-started).
- [autorest](https://github.com/Azure/autorest/tree/main/packages/apps/autorest)
- [PowerShell Core](https://github.com/PowerShell/PowerShell#get-powershell)
- [goimports](https://pkg.go.dev/golang.org/x/tools/cmd/goimports)

# Building

## Generating from TypeSpec

The `Client` is primarily generated from TypeSpec, with some handwritten code where we've changed the interface to match Azure naming conventions (for instance, we refer to Models as Deployments). Files that do not have `custom` (ex: `client.go`, `models.go`, `models_serde.go`, etc..) are generated.

Files that have `custom` in the name are handwritten (ex: `custom_client_files.go`), while files that do not (ex: `client.go`, `models.go`, `models_serde.go`, etc..) are generated.

### Regeneration

The `testdata/tsp-location.yaml` specifies the specific revision (and repo) that we use to generate the client. This also makes it possible, if needed, to generate from branch commmits in [`Azure/azure-rest-api-specs`](https://github.com/Azure/azure-rest-api-specs).

**tsp.location.yaml**:
```yaml
directory: specification/ai/OpenAI.Assistants
commit: <git commit>
repo: Azure/azure-rest-api-specs
```
The generation process is all done as `go generate` commands in `build.go`. To regenerate the client run:

```
go generate ./...
```

Commit the generated changes as part of your pull request.

If the changes don't look quite right you can adjust the generated code using the `autorest.md` file.

# Testing

There are three kinds of tests for this package: unit tests, recorded tests and live tests.

## Unit and recorded tests

Unit tests and recorded tests do not require access to OpenAI to run and will run with any PR as a check-in gate.

Recorded tests require the Azure SDK test proxy is running. See the instructions for [installing the test-proxy](https://github.com/Azure/azure-sdk-tools/blob/main/tools/test-proxy/Azure.Sdk.Tools.TestProxy/README.md#installation).

In one terminal window, start the test-proxy:

```bash
cd <root of the azopenaiassistants module>
test-proxy
```

In another terminal window:


To playback (ie: use recordings):
```bash
cd <root of the azopenaiassistants module>

export AZURE_RECORD_MODE=playback
go test -count 1 -v ./...
```

To re-record:
```bash
cd <root of the azopenaiassistants module>

export AZURE_RECORD_MODE=record
go test -count 1 -v ./...

# push the recording changes to the repo
test-proxy push -a assets.json

# commit our assets.json file now that it points
# to the new recordings.
git add assets.json
git commit -m "updated recordings"
git push
```

## Live tests

### Local development

Copy the `sample.env` file to `.env`, and fill out all the values. Each value is documented to give you a general idea of what's needed, but ultimately you'll need to work with the Azure OpenAI SDK team to figure out which services are used for which features.

Once filled out, the tests will automatically load environment variables from the `.env`:

```bash
export AZURE_RECORD_MODE=live
go test -count 1 -v ./...
```

### Pull requests

Post a comment to your PR with this text:

```
/azp run go - azopenaiassistants
```

The build bot will post a comment indicating its started the pipeline and the checks will start showing up in the status for the PR as well.
21 changes: 21 additions & 0 deletions sdk/ai/azopenaiassistants/LICENSE.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
MIT License

Copyright (c) Microsoft Corporation. All rights reserved.

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE
114 changes: 114 additions & 0 deletions sdk/ai/azopenaiassistants/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,114 @@
# Azure OpenAI assistants client module for Go

NOTE: this client can be used with Azure OpenAI and OpenAI.

OpenAI assistants makes it simpler to have a create, manage and use Assistant, where conversation state is stored and managed by the service. These assistants are backed by the same powerful models you're used to with OpenAI, and also allows the use of the Code Interpreter, Retrieval and Function Calling tools.

Use this module to:

- Create and manage assistants, threads, messages, and runs.
- Configure and use tools with assistants.
- Upload and manage files for use with assistants.

[Source code][azopenaiassistants_repo] | [Package (pkg.go.dev)][azopenaiassistants_pkg_go] | [REST API documentation][openai_rest_docs] | [Product documentation][openai_docs]

## Getting started

### Prerequisites

* Go, version 1.18 or higher - [Install Go](https://go.dev/doc/install)
* [Azure subscription][azure_sub]
* [Azure OpenAI access][azure_openai_access]

### Install the packages

Install the `azopenaiassistants` and `azidentity` modules with `go get`:

```bash
go get github.com/Azure/azure-sdk-for-go/sdk/ai/azopenaiassistants

# optional
go get github.com/Azure/azure-sdk-for-go/sdk/azidentity
```

The [azidentity][azure_identity] module is used for Azure Active Directory authentication with Azure OpenAI.

### Authentication

#### Azure OpenAI

Azure OpenAI clients can authenticate using Azure Active Directory or with an API key:

* Using Azure Active Directory, with a TokenCredential: [example][azopenaiassistants_example_tokencredential]
* Using an API key: [example][azopenaiassistants_example_keycredential]

#### OpenAI

OpenAI supports connecting using an API key: [example][azopenaiassistants_example_openai].

## Key concepts

See [Key concepts][openai_key_concepts_assistants] in the product documentation for more details about general concepts.

# Examples

Examples for various scenarios can be found on [pkg.go.dev][azopenaiassistants_examples] or in the example*_test.go files in our GitHub repo for [azopenaiassistants][azopenaiassistants_github].

## Troubleshooting

### Error Handling

All methods that send HTTP requests return `*azcore.ResponseError` when these requests fail. `ResponseError` has error details and the raw response from the service.

### Logging

This module uses the logging implementation in `azcore`. To turn on logging for all Azure SDK modules, set `AZURE_SDK_GO_LOGGING` to `all`. By default, the logger writes to stderr. Use the `azcore/log` package to control log output. For example, logging only HTTP request and response events, and printing them to stdout:

```go
import azlog "github.com/Azure/azure-sdk-for-go/sdk/azcore/log"

// Print log events to stdout
azlog.SetListener(func(cls azlog.Event, msg string) {
fmt.Println(msg)
})

// Includes only requests and responses in credential logs
azlog.SetEvents(azlog.EventRequest, azlog.EventResponse)
```

## Contributing

This project welcomes contributions and suggestions. Most contributions require you to agree to a [Contributor License Agreement (CLA)][cla] declaring that you have the right to, and actually do, grant us the rights to use your contribution.

When you submit a pull request, a CLA-bot will automatically determine whether you need to provide a CLA and decorate
the PR appropriately (e.g., label, comment). Simply follow the instructions provided by the bot. You will only need to
do this once across all repos using our CLA.

This project has adopted the [Microsoft Open Source Code of Conduct][coc]. For more information, see
the [Code of Conduct FAQ][coc_faq] or contact [[email protected]][coc_contact] with any additional questions or
comments.

<!-- LINKS -->
[azure_openai_access]: https://learn.microsoft.com/azure/cognitive-services/openai/overview#how-do-i-get-access-to-azure-openai
[azopenaiassistants_repo]: https://github.com/Azure/azure-sdk-for-go/tree/main/sdk/ai/azopenaiassistants

<!-- TODO: BEGIN: will replace once the link is actually available. -->
[azopenaiassistants_pkg_go]: https://pkg.go.dev/github.com/Azure/azure-sdk-for-go/sdk/ai/azopenai
[azopenaiassistants_examples]: https://pkg.go.dev/github.com/Azure/azure-sdk-for-go/sdk/ai/azopenai#pkg-examples
[azopenaiassistants_example_tokencredential]: https://pkg.go.dev/github.com/Azure/azure-sdk-for-go/sdk/ai/azopenai#example-NewClient
[azopenaiassistants_example_keycredential]: https://pkg.go.dev/github.com/Azure/azure-sdk-for-go/sdk/ai/azopenai#example-NewClientWithKeyCredential
[azopenaiassistants_example_openai]: https://pkg.go.dev/github.com/Azure/azure-sdk-for-go/sdk/ai/azopenai#example-NewClientForOpenAI
[azopenaiassistants_github]: https://github.com/Azure/azure-sdk-for-go/blob/main/sdk/ai/azopenai
<!-- TODO: END: will replace once the link is actually available. -->

[azure_identity]: https://pkg.go.dev/github.com/Azure/azure-sdk-for-go/sdk/azidentity
[azure_sub]: https://azure.microsoft.com/free/
[openai_docs]: https://learn.microsoft.com/azure/cognitive-services/openai
[openai_key_concepts]: https://learn.microsoft.com/azure/cognitive-services/openai/overview#key-concepts
[openai_key_concepts_assistants]: https://platform.openai.com/docs/assistants/overview
[openai_rest_docs]: https://learn.microsoft.com/azure/cognitive-services/openai/reference
[cla]: https://cla.microsoft.com
[coc]: https://opensource.microsoft.com/codeofconduct/
[coc_faq]: https://opensource.microsoft.com/codeofconduct/faq/
[coc_contact]: mailto:[email protected]
[azure_openai_quickstart]: https://learn.microsoft.com/azure/cognitive-services/openai/quickstart
6 changes: 6 additions & 0 deletions sdk/ai/azopenaiassistants/assets.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
{
"AssetsRepo": "Azure/azure-sdk-assets",
"AssetsRepoPrefixPath": "go",
"TagPrefix": "go/ai/azopenaiassistants",
"Tag": "go/ai/azopenaiassistants_eee72902cb"
}
33 changes: 33 additions & 0 deletions sdk/ai/azopenaiassistants/autorest.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
# Go

These settings apply only when `--go` is specified on the command line.

``` yaml
input-file:
# PR: https://github.com/Azure/azure-rest-api-specs/pull/27076/files
#- https://raw.githubusercontent.com/Azure/azure-rest-api-specs/18c24352ad4a2e0959c0b4ec1404c3a250912f8b/specification/ai/data-plane/OpenAI.Assistants/OpenApiV2/preview/2024-02-15-preview/assistants_generated.json
- ./testdata/generated/openapi.json
output-folder: ../azopenaiassistants
clear-output-folder: false
module: github.com/Azure/azure-sdk-for-go/sdk/ai/azopenaiassistants
license-header: MICROSOFT_MIT_NO_VERSION
openapi-type: data-plane
go: true
title: "OpenAIAssistants"
use: "@autorest/[email protected]"
slice-elements-byval: true
# can't use this since it removes an innererror type that we want ()
# remove-non-reference-schema: true
```

## Transformations

Fix deployment and endpoint parameters so they show up in the right spots

``` yaml
directive:
# Add x-ms-parameter-location to parameters in x-ms-parameterized-host
- from: swagger-document
where: $["x-ms-parameterized-host"].parameters.0
transform: $["x-ms-parameter-location"] = "client";
```
13 changes: 13 additions & 0 deletions sdk/ai/azopenaiassistants/build.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
//go:build go1.18
// +build go1.18

// Copyright (c) Microsoft Corporation. All rights reserved.
// Licensed under the MIT License. See License.txt in the project root for license information.

package azopenaiassistants

//go:generate pwsh ./testdata/genopenapi.ps1
//go:generate go run ./internal/transform

//go:generate goimports -w ./..
//go:generate go mod tidy
46 changes: 46 additions & 0 deletions sdk/ai/azopenaiassistants/ci.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
# NOTE: Please refer to https://aka.ms/azsdk/engsys/ci-yaml before editing this file.
trigger:
branches:
include:
- main
- feature/*
- hotfix/*
- release/*
paths:
include:
- sdk/ai/azopenaiassistants
- eng/

pr:
branches:
include:
- main
- feature/*
- hotfix/*
- release/*
paths:
include:
- sdk/ai/azopenaiassistants

stages:
- template: /eng/pipelines/templates/jobs/archetype-sdk-client.yml
parameters:
# We need to allow for longer retry times with tests that run against the public endpoint
# which throttles under load. Note, I left a little wiggle room since the TimeoutInMinutes
# controls the overall pipeline and TestRunTime configures the individual `go test -timeout` parameter.
TimeoutInMinutes: 35
TestRunTime: 30m
ServiceDirectory: "ai/azopenaiassistants"
RunLiveTests: true
UsePipelineProxy: false
CloudConfig:
Public:
SubscriptionConfigurations:
- $(sub-config-azure-cloud-test-resources)
- $(sub-config-openai-test-resources) # TestSecrets-openai
EnvVars:
AZURE_TEST_RUN_LIVE: "true" # use when utilizing the New-TestResources Script
AZURE_CLIENT_ID: $(AZOPENAIASSISTANTS_CLIENT_ID)
AZURE_CLIENT_SECRET: $(AZOPENAIASSISTANTS_CLIENT_SECRET)
AZURE_TENANT_ID: $(AZOPENAIASSISTANTS_TENANT_ID)
AZURE_SUBSCRIPTION_ID: $(AZOPENAIASSISTANTS_SUBSCRIPTION_ID)
Loading