Skip to content

Conversation

@falonso81
Copy link
Contributor

@falonso81 falonso81 commented Dec 10, 2024

Add context caching to vertexai
#1443

How to test:

func main() {
    fmt.Println("hello world!!!!")
    ctx := context.Background()
    if err := vertexai.Init(ctx, nil); err != nil {
        log.Fatal(err)
    }
    m := vertexai.Model("gemini-1.5-flash-002")
    if m == nil {
        log.Fatal(errors.New("vertexai init failed"))
    }

    strrr, err := ai.GenerateText(ctx, m, ai.WithTextPrompt("Tell me a joke about golang developers."))
    if err != nil {
        fmt.Println(err)
    }
    fmt.Println(strrr)

    strrr, err = ai.GenerateText(ctx, m, ai.WithTextPrompt("Tell me a joke about golang developers. "))
    if err != nil {
        fmt.Println(err)
    }
    fmt.Println(strrr)
}

Checklist (if applicable):

@falonso81 falonso81 changed the title feat(go): add context caching to vertexai Dec 10, 2024
"gemini-1.5-flash": gemini.Multimodal,
"gemini-1.0-pro": gemini.BasicText,
"gemini-1.5-pro": gemini.Multimodal,
"gemini-1.5-flash-002": gemini.Multimodal,
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

context caching is working only with stable Gemini model versions using number prefix like https://cloud.google.com/vertex-ai/generative-ai/docs/learn/model-versions#stable-versions-available

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We don't want to get rid of the plain gemini-1.5-flash. @hugoaguirre is working on adding model version support per model family, please work with him on how combine the two efforts.

@falonso81 falonso81 requested a review from apascal07 December 10, 2024 14:45
@falonso81 falonso81 linked an issue Dec 13, 2024 that may be closed by this pull request
@apascal07
Copy link
Collaborator

Please add tests for the caching business logic and a sample where this is used (by setting the cache TTL, which enables context caching). See #1297 for test coverage.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Go style guide uses tabs (as 4 spaces) instead of 2 spaces. Please update here and in the other PR.

"gemini-1.5-flash": gemini.Multimodal,
"gemini-1.0-pro": gemini.BasicText,
"gemini-1.5-pro": gemini.Multimodal,
"gemini-1.5-flash-002": gemini.Multimodal,
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We don't want to get rid of the plain gemini-1.5-flash. @hugoaguirre is working on adding model version support per model family, please work with him on how combine the two efforts.

@falonso81 falonso81 force-pushed the 1443-add-context-caching-to-vertex-ai branch from 8562c77 to 7ef5090 Compare December 17, 2024 17:57
@falonso81
Copy link
Contributor Author

closing , new PR here #1566

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

2 participants