[FirebaseAI] Implicit caching support

[paulb777]

Add metadata support for the two fields related to implicit caching.

Summary of Changes

This pull request introduces comprehensive support for implicit caching within the FirebaseAI library. It enables the logging and retrieval of token usage metadata related to cached content, providing developers with insights into the efficiency of their generative model calls. The changes include adding new fields to the GenerateContentResponse structure, updating its decoding logic, and implementing dedicated integration and unit tests to validate the new functionality.

Highlights

• Implicit Caching Support: Introduced support for implicit caching metadata in GenerateContentResponse, allowing access to cachedContentTokenCount and cacheTokensDetails within UsageMetadata to track token savings from cached content.
• New Usage Metadata Fields: Added cachedContentTokenCount (Int) and cacheTokensDetails ([ModalityTokenCount]) properties to the GenerateContentResponse.UsageMetadata struct and updated its Decodable conformance to correctly parse these new fields.
• Integration Tests for Caching: Included a new integration test suite (ImplicitCacheTests.swift) to specifically verify the functionality and reporting of implicit caching, particularly for scenarios involving large contexts.
• Updated Unit Tests: Enhanced existing unit tests in APITests.swift to ensure correct decoding and handling of the new cachedContentTokenCount and cacheTokensDetails fields in GenerateContentResponse.
• Changelog Update: Documented the new implicit caching feature and its associated metadata fields in the CHANGELOG.md file.

[gemini-code-assist]

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point in your pull request via creating an issue comment (i.e. comment on the pull request page) using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in issue comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist is currently in preview and may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments to provide feedback.

[paulb777]

/gemini summarize

[paulb777]

/gemini review

[gemini-code-assist]

Hello paulb777,

I understand you're looking for a code review. The /gemini review command is designed to initiate a comprehensive review of the entire pull request, which will be posted as a new top-level comment.

In this conversation thread, I'm here to answer specific questions you might have about the code or previous review comments, or to provide targeted suggestions.

If you'd like a full review of the pull request, please create a new comment with /gemini review (or /gemini summarize for a summary) and I will generate it as a separate, new comment on the pull request.

[gemini-code-assist]

Code Review

This pull request adds support for implicit caching in GenerateContentResponse by introducing cachedContentTokenCount and cacheTokensDetails to UsageMetadata. The changes are well-implemented, including updates to the data model, decoding logic, and comprehensive tests. I've included a few minor suggestions to improve test robustness, code style, and documentation clarity. Overall, this is a solid contribution.

[paulb777]

/gemini summarize

[gemini-code-assist]

Summary of Changes

This pull request introduces comprehensive support for implicit caching within the FirebaseAI library. It enables the logging and retrieval of token usage metadata related to cached content, providing developers with insights into the efficiency of their generative model calls. The changes include adding new fields to the GenerateContentResponse structure, updating its decoding logic, and implementing dedicated integration and unit tests to validate the new functionality.

Highlights

• Implicit Caching Support: Introduced support for implicit caching metadata in GenerateContentResponse, allowing access to cachedContentTokenCount and cacheTokensDetails within UsageMetadata to track token savings from cached content.
• New Usage Metadata Fields: Added cachedContentTokenCount (Int) and cacheTokensDetails ([ModalityTokenCount]) properties to the GenerateContentResponse.UsageMetadata struct and updated its Decodable conformance to correctly parse these new fields.
• Integration Tests for Caching: Included a new integration test suite (ImplicitCacheTests.swift) to specifically verify the functionality and reporting of implicit caching, particularly for scenarios involving large contexts.
• Updated Unit Tests: Enhanced existing unit tests in APITests.swift to ensure correct decoding and handling of the new cachedContentTokenCount and cacheTokensDetails fields in GenerateContentResponse.
• Changelog Update: Documented the new implicit caching feature and its associated metadata fields in the CHANGELOG.md file.

🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console.

Changelog

• FirebaseAI/CHANGELOG.md
  • Added a new unreleased feature entry detailing the support for implicit caching metadata in GenerateContentResponse, including cachedContentTokenCount and cacheTokensDetails in UsageMetadata.
• FirebaseAI/Sources/GenerateContentResponse.swift
  • Introduced cachedContentTokenCount and cacheTokensDetails properties to the UsageMetadata struct.
  • Updated the Decodable conformance for UsageMetadata to correctly decode these new fields from the API response.
• FirebaseAI/Tests/TestApp/Tests/Integration/GenerateContentIntegrationTests.swift
  • Added assertions to existing integration tests to confirm that cachedContentTokenCount is 0 and cacheTokensDetails is empty when implicit caching is not expected to be active.
• FirebaseAI/Tests/TestApp/Tests/Integration/ImplicitCacheTests.swift
  • Added a new test file containing ImplicitCacheTests to specifically verify implicit caching behavior.
  • Includes a test case that sends a large prompt twice to trigger and assert on cachedContentTokenCount and cacheTokensDetails values.
• FirebaseAI/Tests/Unit/APITests.swift
  • Added a new unit test testGenerateContentResponseWithCacheMetadata to confirm the correct decoding of GenerateContentResponse when cachedContentTokenCount and cacheTokensDetails are present in the JSON payload.

Activity

• gemini-code-assist[bot] posted an initial message about using Gemini Code Assist.
• paulb777 requested a summary (/gemini summarize).
• paulb777 requested a review (/gemini review).
• gemini-code-assist[bot] clarified the usage of /gemini review for full PR reviews versus specific questions.
• paulb777 requested a summary again (/gemini summarize).
• gemini-code-assist[bot] provided a review comment suggesting a minor typo fix in a documentation comment (cachedContent to cached content).
• gemini-code-assist[bot] provided a review comment recommending a more robust assertion for cacheTokensDetails by summing tokenCount from all elements instead of assuming a single element.
• gemini-code-assist[bot] provided a review comment pointing out a minor indentation style fix in a JSON string within a test.