Merge branch 'main' into gemini-magic

Author: MarkDaoust

CONTRIBUTING.md

@@ -24,7 +24,7 @@ Follow either of the two links above to access the appropriate CLA and
 instructions for how to sign and return it. Once we receive it, we'll be able to
 accept your pull requests.
 
-## Contributing A Patch
+## Contributing a Patch
 
 1.  Submit an issue describing your proposed change to the repo in question.
 1.  The repo owner will respond to your issue promptly.
@@ -45,7 +45,7 @@ accept your pull requests.
 1.  [Set up authentication with a service account][auth] so you can access the
     API from your local workstation.
 
-You can use an API-key, but remember never to same it in your source files.
+You can use an API-key, but remember never to save it in your source files.
 
 
 ## Development
@@ -103,4 +103,4 @@ python docs/build_docs.py
 [projects]: https://console.cloud.google.com/project
 [billing]: https://support.google.com/cloud/answer/6293499#enable-billing
 [enable_api]: https://console.cloud.google.com/flows/enableapi?apiid=generativelanguage.googleapis.com
-[auth]: https://cloud.google.com/docs/authentication/getting-started
+[auth]: https://cloud.google.com/docs/authentication/getting-started

README.md

@@ -1,47 +1,146 @@
-Google Generative AI Python Client
-==================================
+# Google AI Python SDK
 
 [![PyPI version](https://badge.fury.io/py/google-generativeai.svg)](https://badge.fury.io/py/google-generativeai)
 ![Python support](https://img.shields.io/pypi/pyversions/google-generativeai)
 ![PyPI - Downloads](https://img.shields.io/pypi/dd/google-generativeai)
 
-Get started using the PaLM API in Python. Check out the [developer site](https://developers.generativeai.google/)
-for comprehensive documentation.
+The Google AI Python SDK enables developers to use Google's state-of-the-art generative AI
+models (like Gemini and PaLM) to build AI-powered features and applications. This SDK
+supports use cases like:
+
+- Generate text from text-only input
+- Generate text from text-and-images input (multimodal) (for Gemini only)
+- Build multi-turn conversations (chat)
+- Embedding
+
+For example, with just a few lines of code, you can access Gemini's multimodal
+capabilities to generate text from text-and-image input:
+
+```
+model = genai.GenerativeModel('gemini-pro-vision')
+
+cookie_picture = {
+    'mime_type': 'image/png',
+    'data': Path('cookie.png').read_bytes()
+}
+prompt = "Give me a recipe for this:"
+
+response = model.generate_content(
+    content=[prompt, cookie_picture]
+)
+print(response.text)
+```
 
-## Installation and usage
+
+## Try out the API
 
 Install from PyPI.
-```bash
-pip install google-generativeai
+
+`pip install google-generativeai`
+
+[Obtain an API key from AI Studio](https://makersuite.google.com/app/apikey),
+then configure it here.
+
+Import the SDK and load a model.
+
 ```
+import google.generativeai as genai
 
-Get an [API key from MakerSuite](https://makersuite.google.com/app/apikey), then configure it here.
-```python
+genai.configure(api_key=os.environ["API_KEY"])
+
+model = genai.GenerativeModel('gemini-pro')
+```
+
+Use `GenerativeModel.generate_content` to have the model complete some initial text.
+
+```
+response = model.generate_content("The opposite of hot is")
+print(response.text)  # cold.
+```
+
+Use `GenerativeModel.start_chat` to have a discussion with a model.
+
+```
+chat = model.start_chat()
+response = chat.send_message('Hello, what should I have for dinner?')
+print(response.text) #  'Here are some suggestions...'
+response = chat.send_message("How do I cook the first one?")
+```
+
+
+
+## Installation and usage
+
+Run [`pip install google-generativeai`](https://pypi.org/project/google-generativeai).
+
+For detailed instructions, you can find a
+[quickstart](https://ai.google.dev/tutorials/python_quickstart) for the Google AI
+Python SDK in the Google documentation.
+
+This quickstart describes how to add your API key and install the SDK in your app,
+initialize the model, and then call the API to access the model. It also describes some
+additional use cases and features, like streaming, embedding, counting tokens, and
+controlling responses.
+
+
+## Documentation
+
+Find complete documentation for the Google AI SDKs and the Gemini model in the Google
+documentation: https://ai.google.dev/docs
+
+
+## Contributing
+
+See [Contributing](https://github.com/google/generative-ai-python/blob/main/CONTRIBUTING.md) for more information on contributing to the Google AI Python SDK.
+
+## Developers who use the PaLM API
+
+### Migrate to use the Gemini API
+
+Check our [migration guide](https://ai.google.dev/docs/migration_guide) in the Google
+documentation.
+
+### Installation and usage for the PaLM API
+
+Install from PyPI.
+
+`pip install google-generativeai`
+
+[Obtain an API key from AI Studio](https://makersuite.google.com/app/apikey), then
+configure it here.
+
+```
 import google.generativeai as palm
 
 palm.configure(api_key=os.environ["PALM_API_KEY"])
 ```
 
-Use [`palm.generate_text`](https://developers.generativeai.google/api/python/google/generativeai/generate_text)
-to have the model complete some initial text.
-```python
+Use `palm.generate_text` to have the model complete some initial text.
+
+```
 response = palm.generate_text(prompt="The opposite of hot is")
 print(response.result)  # cold.
 ```
 
-Use [`palm.chat`](https://developers.generativeai.google/api/python/google/generativeai/chat)
-to have a discussion with a model.
-```python
+Use `palm.chat` to have a discussion with a model.
+
+```
 response = palm.chat(messages=["Hello."])
 print(response.last) #  'Hello! What can I help you with?'
 response.reply("Can you tell me a joke?")
 ```
 
-## Documentation
+### Documentation for the PaLM API
+
+- [General PaLM documentation](https://ai.google.dev/docs/palm_api_overview)
+
+- [Text quickstart](https://github.com/google/generative-ai-docs/blob/main/site/en/palm_docs/text_quickstart.ipynb)
 
-Checkout the full [API docs](https://developers.generativeai.google/api), the [guide](https://developers.generativeai.google/guide) and [quick starts](https://developers.generativeai.google/tutorials).
+- [Chat quickstart](https://github.com/google/generative-ai-docs/blob/main/site/en/palm_docs/chat_quickstart.ipynb)
 
-## Colab magics
+- [Tuning quickstart](https://github.com/google/generative-ai-docs/blob/main/site/en/palm_docs/tuning_quickstart_python.ipynb)
+
+### Colab magics
 
 ```
 %pip install -q google-generativeai
@@ -54,3 +153,7 @@ Once installed, use the Python client via the `%%llm` Colab magic. Read the full
 %%llm
 The best thing since sliced bread is
 ```
+
+## License
+
+The contents of this repository are licensed under the [Apache License, version 2.0](http://www.apache.org/licenses/LICENSE-2.0).

docs/build_docs.py

@@ -24,6 +24,10 @@
 import pathlib
 import re
 import textwrap
+import typing
+
+# For showing the conditional imports and types in `content_types.py`
+typing.TYPE_CHECKING = True
 
 from absl import app
 from absl import flags

google/generativeai/__init__.py

@@ -12,60 +12,31 @@
 # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 # See the License for the specific language governing permissions and
 # limitations under the License.
-"""A high level client library for generative AI.
+"""Google AI Python SDK
 
 ## Setup
 
 ```posix-terminal
 pip install google-generativeai
 ```
 
-```
-import google.generativeai as palm
-import os
-
-palm.configure(api_key=os.environ['API_KEY'])
-```
-
-## Text
-
-Use the `palm.generate_text` function to have the model complete some initial
-text.
-
-```
-response = palm.generate_text(prompt="The opposite of hot is")
-print(response.result) #  'cold.'
-```
+## GenerativeModel
 
-## Chat
+Use `genai.GenerativeModel` to access the API:
 
-Use the `palm.chat` function to have a discussion with a model:
-
-```
-chat = palm.chat(messages=["Hello."])
-print(chat.last) #  'Hello! What can I help you with?'
-chat = chat.reply("Can you tell me a joke?")
-print(chat.last) #  'Why did the chicken cross the road?'
 ```
+import google.generativeai as genai
+import os
 
-## Models
-
-Use the model service discover models and find out more about them:
-
-Use `palm.get_model` to get details if you know a model's name:
-
-```
-model = palm.get_model('models/chat-bison-001') # 🦬
-```
+genai.configure(api_key=os.environ['API_KEY'])
 
-Use `palm.list_models` to discover models:
+model = genai.GenerativeModel(name='gemini-pro')
+response = model.generate_content('Please summarise this document: ...')
 
-```
-import pprint
-for model in palm.list_models():
-    pprint.pprint(model) # 🦎🦦🦬🦄
+print(response.text)
 ```
 
+See the [python quickstart](https://ai.google.dev/tutorials/python_quickstart) for more details.
 """
 from __future__ import annotations
 
@@ -82,6 +53,7 @@
 from google.generativeai.embedding import embed_content
 
 from google.generativeai.generative_models import GenerativeModel
+from google.generativeai.generative_models import ChatSession
 
 from google.generativeai.text import generate_text
 from google.generativeai.text import generate_embeddings

google/generativeai/generative_models.py

@@ -2,26 +2,26 @@
 
 from __future__ import annotations
 
+from collections.abc import Iterable
 import dataclasses
 import textwrap
+from typing import Union
 
 # pylint: disable=bad-continuation, line-too-long
 
 
-from collections.abc import Iterable
-
 from google.ai import generativelanguage as glm
 from google.generativeai import client
 from google.generativeai import string_utils
 from google.generativeai.types import content_types
 from google.generativeai.types import generation_types
 from google.generativeai.types import safety_types
 
-_GENERATE_CONTENT_ASYNC_DOC = """The async version of `Model.generate_content`."""
+_GENERATE_CONTENT_ASYNC_DOC = """The async version of `GenerativeModel.generate_content`."""
 
 _GENERATE_CONTENT_DOC = """A multipurpose function to generate responses from the model.
 
-This `GenerativeModel.generate_content` method can handle multimodal input, and multiturn
+This `GenerativeModel.generate_content` method can handle multimodal input, and multi-turn
 conversations.
 
 >>> model = genai.GenerativeModel('models/gemini-pro')
@@ -70,6 +70,7 @@
     generation_config: Overrides for the model's generation config.
     safety_settings: Overrides for the model's safety settings.
     stream: If True, yield response chunks as they are generated. 
+    tools: `glm.Tools` more info coming soon.
 """
 
 _SEND_MESSAGE_ASYNC_DOC = """The async version of `ChatSession.send_message`."""
@@ -158,6 +159,7 @@ def __init__(
         model_name: str = "gemini-m",
         safety_settings: safety_types.SafetySettingOptions | None = None,
         generation_config: generation_types.GenerationConfigType | None = None,
+        tools: content_types.ToolsType = None,
     ):
         if "/" not in model_name:
             model_name = "models/" + model_name
@@ -166,6 +168,8 @@ def __init__(
             safety_settings, harm_category_set="new"
         )
         self._generation_config = generation_types.to_generation_config_dict(generation_config)
+        self._tools = content_types.to_tools(tools)
+
         self._client = None
         self._async_client = None
 
@@ -213,6 +217,7 @@ def _prepare_request(
             contents=contents,
             generation_config=merged_gc,
             safety_settings=merged_ss,
+            tools=self._tools,
             **kwargs,
         )
 
@@ -274,21 +279,34 @@ async def generate_content_async(
     def count_tokens(
         self, contents: content_types.ContentsType
     ) -> glm.CountTokensResponse:
+        if self._client is None:
+            self._client = client.get_default_generative_client()
         contents = content_types.to_contents(contents)
-        return self._client.count_tokens(self.model_name, contents)
+        return self._client.count_tokens(glm.CountTokensRequest(model=self.model_name, contents=contents))
 
     async def count_tokens_async(
         self, contents: content_types.ContentsType
     ) -> glm.CountTokensResponse:
+        if self._async_client is None:
+            self._async_client = client.get_default_generative_async_client()
         contents = content_types.to_contents(contents)
-        return await self._client.count_tokens(self.model_name, contents)
+        return await self._async_client.count_tokens(glm.CountTokensRequest(model=self.model_name, contents=contents))
     # fmt: on
 
     def start_chat(
         self,
         *,
         history: Iterable[content_types.StrictContentType] | None = None,
     ) -> ChatSession:
+        """Returns a `genai.ChatSession` attached to this model.
+
+        >>> model = genai.GenerativeModel()
+        >>> chat = model.start_chat(history=[...])
+        >>> response = chat.send_message("Hello?")
+
+        Arguments:
+            history: An iterable of `glm.Content` objects, or equvalents to initialize the session.
+        """
         if self._generation_config.get("candidate_count", 1) > 1:
             raise ValueError("Can't chat with `candidate_count > 1`")
         return ChatSession(