Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

googleai: return err not log.Fatal when stream get error #663

Merged
merged 1 commit into from
Mar 11, 2024

Conversation

Abirdcfly
Copy link
Contributor

the log.Fatal will print the error and exit the program. https://pkg.go.dev/log#Fatal

Fatal is equivalent to Print followed by a call to os.Exit(1).

But the user may have written a series of error handling functions, and I think we should throw the error to the user to deal with, rather than forcing the user's program to stop.

One error that can be reproduced manually is when the request rate limit of the gemini is exceeded.

example: (https://github.com/Abirdcfly/langchaingo-google-stream-exit-early)

// Set the API_KEY env var to your API key taken from ai.google.dev
package main

import (
	"context"
	"fmt"
	"log"
	"os"

	"github.com/tmc/langchaingo/llms"
	"github.com/tmc/langchaingo/llms/googleai"
)

func main() {
	ctx := context.Background()
	apiKey := os.Getenv("API_KEY")
	llm, err := googleai.New(ctx, googleai.WithAPIKey(apiKey))
	if err != nil {
		log.Fatalf("Failed to create client: %v", err)
	}

	prompts := make([]string, 60)
	for i := 1; i <= 60; i++ {
		prompts[i-1] = fmt.Sprintf("Who was the %d person to walk on the moon?", i)
	}
	for _, prompt := range prompts {
		fmt.Printf("\nHuman:\t%s\nstream:\t", prompt)
		answer, err := llms.GenerateFromSinglePrompt(ctx, llm, prompt, llms.WithStreamingFunc(func(ctx context.Context, chunk []byte) error {
			fmt.Print(string(chunk) + ".")
			return nil
		}))
		if err != nil {
			log.Fatalf("Failed to generate: %v, the prompt is:%s", err, prompt)
		}
		fmt.Printf("\nAI:\t%s\n", answer)
	}
}

when before current pull request: (https://github.com/Abirdcfly/langchaingo-google-stream-exit-early/actions/runs/8228267597/job/22497418099#step:4:516)

Human:	Who was the 23 person to walk on the moon?
2024/03/11 05:50:50 googleapi: Error 500:
exit status 1
stream:	

when after current pull request:(https://github.com/Abirdcfly/langchaingo-google-stream-exit-early/actions/runs/8228342012/job/22497623326#step:4:428)

Human:	Who was the 1 person to walk on the moon?
2024/03/11 06:01:40 Failed to generate: error in stream mode: googleapi: Error 500:, the prompt is:Who was the 1 person to walk on the moon?
exit status 1
stream:	

@Abirdcfly
Copy link
Contributor Author

cc @eliben @tmc

Copy link
Collaborator

@eliben eliben left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks

@eliben eliben merged commit 24cb833 into tmc:main Mar 11, 2024
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants