Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat(vertexai): explicit caching #10310

Closed
wants to merge 10 commits into from
Closed

feat(vertexai): explicit caching #10310

wants to merge 10 commits into from

Conversation

jba
Copy link
Contributor

@jba jba commented Jun 4, 2024 •
edited
Loading

Implement explicit caching.

  • The CachedContent type.

  • Methods to create, update, list and delete CachedContents.
    The update method follows the same pattern in some other cloud clients,
    where a separate type carries the fields to be updated.

  • Two ways to set the CachedContent in a model:

    1. The GenerativeModel.CachedContentName field, which corresponds
      to the actual proto.
    2. The GenerativeModelFromCachedContent method, which requires
      a properly initialized CachedContent but guarantees that
      the model names of the CachedContent and GenerativeModel are
      the same.

@jba jba requested a review from eliben June 4, 2024 12:31
@jba jba requested review from a team as code owners June 4, 2024 12:31
@jba jba force-pushed the jba-vertex-caching branch 2 times, most recently from b3718aa to 77025ce Compare June 4, 2024 12:41
@jba
Copy link
Contributor Author

jba commented Jun 4, 2024 •
edited
Loading

OK TO SUBMIT. Feature is now available.

eliben
eliben reviewed Jun 4, 2024
View reviewed changes
vertexai/genai/client.go Show resolved Hide resolved
vertexai/genai/caching.go Show resolved Hide resolved
eliben
eliben approved these changes Jun 5, 2024
View reviewed changes
vertexai/genai/client.go Show resolved Hide resolved
vertexai/genai/example_test.go Outdated Show resolved Hide resolved
vertexai/genai/caching.go Show resolved Hide resolved
@jba jba enabled auto-merge (squash) June 10, 2024 16:36
@jba jba force-pushed the jba-vertex-caching branch from 5a34c41 to 46f5dae Compare June 10, 2024 16:48
jba added 10 commits June 10, 2024 14:59
@jba
feat(vertexai): explicit caching
d5eee44 
Implement explicit caching.

- The CachedContent type.

- Client methods to create, update, list and delete CachedContents.
  The update method follows the same pattern in some other cloud clients,
  where a separate type carries the fields to be updated.

- Two ways to set the CachedContent in a model:
  1. The GenerativeModel.CachedContentName field, which corresponds
     to the actual proto.
  2. The GenerativeModelFromCachedContent method, which requires
     a properly initialized CachedContent but guarantees that
     the model names of the CachedContent and GenerativeModel are
     the same.
@jba
fix .gen.go
58a1cc6
@jba
tests
488d2c8
@jba
generate test
52d9c04
@jba
add examples, test update
b037a92
@jba
fix TTL test, reviewer comments
5588306
@jba
reviewer comments
fca1783
@jba
switch to standard endpoint
16e20c5
@jba
go mod tidy
7061902
@jba
fix copyright year
a097b46
@jba jba force-pushed the jba-vertex-caching branch from 46f5dae to a097b46 Compare June 10, 2024 19:00
@jba
Copy link
Contributor Author

jba commented Jun 10, 2024

@codyoss Do you see the problem with kokoro?

@codyoss
Copy link
Member

codyoss commented Jun 10, 2024

@jba

genai/aiplatformpb_veneer.gen.go:123:39: undefined: pb.CachedContent
genai/aiplatformpb_veneer.gen.go:127:11: undefined: pb.CachedContent
genai/aiplatformpb_veneer.gen.go:141:38: undefined: pb.CachedContent
genai/aiplatformpb_veneer.gen.go:127:11: too many errors
FAIL	cloud.google.com/go/vertexai/genai [build failed]
?   	cloud.google.com/go/vertexai/internal	[no test files]

@eliben
Copy link
Contributor

eliben commented Jun 10, 2024

@jba

genai/aiplatformpb_veneer.gen.go:123:39: undefined: pb.CachedContent
genai/aiplatformpb_veneer.gen.go:127:11: undefined: pb.CachedContent
genai/aiplatformpb_veneer.gen.go:141:38: undefined: pb.CachedContent
genai/aiplatformpb_veneer.gen.go:127:11: too many errors
FAIL	cloud.google.com/go/vertexai/genai [build failed]
?   	cloud.google.com/go/vertexai/internal	[no test files]

@codyoss could you describe the path of mouse clicks taking you to this information from the kokoro "Details" page?

@codyoss
Copy link
Member

codyoss commented Jun 10, 2024

Click the details button next to kokoro:
Screenshot 2024-06-10 at 3 32 39 PM

Copy and paste the link to the failing build:
Screenshot 2024-06-10 at 3 33 03 PM

@jba
Copy link
Contributor Author

jba commented Jun 10, 2024

I guess this builds locally because of go.work.
To get the new symbols we should make a new release of aiplatform.
#9772 is the PR. It should be unblocked. I just approved it but its Do Not Merge label should be removed.

@eliben eliben mentioned this pull request Jun 11, 2024
Merged
@eliben
Copy link
Contributor

eliben commented Jun 11, 2024

Since @jba is now OOO, I sent branched this PR and sent #10363 with the same contents, except updating the aiplatform dependency

gcf-merge-on-green bot pushed a commit that referenced this pull request Jun 11, 2024
@eliben
feat(vertexai): explicit caching (#10363)
d9754c7 
This PR takes over from #10310 and adds the dependency bump for `cloud.google.com/go/aiplatform v1.68.0` following the landing of #9772
@eliben
Copy link
Contributor

eliben commented Jun 11, 2024

Merged in #10363, which was released in v0.11.0

@eliben eliben closed this Jun 11, 2024
auto-merge was automatically disabled June 11, 2024 17:52

Pull request was closed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@eliben eliben eliben approved these changes

Assignees
No one assigned
Labels
breaking change
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@jba @codyoss @eliben