The application demonstrate and IBM Cloud Function (based on Apache OpenWhisk) that connects Facebook Messenger with Watson Assitant, Visual Recognition and Watson Discovery saving the chat history in Cloudant database.
One function, or action, is invoked through a web endpoint provided by IBM Cloud Functions and called by the Facebook Messenger Webhook. The message is sent to Watson Assistant to interact with a virtual agent, if the message is an image its sent to Watson Visual Recognition.
After going through this pattern you will understand how to:
- Use Watson Assistant
- Use Watson Visual Recognition
- Create and Deploy Cloud Functions
- User interacts with Facebook Messenger
- Facebook Messenger sends the payload to IBM Cloud Functions
- The function (or action) looks up for a past chat history on Cloudant Database.
- The function sends the text message to Watson Assistant.
- If need the function will send an attached image to Watson Visual Recognition.
- If need the function will fallback to Watson Discovery to find the answer for the user
- The function saves the chat history to Cloudant Database.
- The function sends the answer to Facebook Messenger.
- The user gets the answer to his interaction.
- Cloudant: A fully managed data layer designed for modern web and mobile applications that leverages a flexible JSON schema.
- Watson Visual Recognition: Visual Recognition service uses deep learning algorithms to identify scenes, objects, and faces in images you upload to the service. You can create and train a custom classifier to identify subjects that suit your needs.
- Watson Assistant: Assistant service combines machine learning, natural language understanding, and integrated dialog tools to create conversation flows between your apps and your users.
- IBM Cloud Functions (powered by Apache OpenWhisk): Execute code on demand in a highly scalable, serverless environment.
- Watson: Watson on the IBM Cloud allows you to integrate the world's most powerful AI into your application and store, train and manage your data in the most secure cloud.
- Serverless: An event-action platform that allows you to execute code in response to an event.
-
IBM Cloud Functions CLI to create cloud functions from the terminal. Make sure you do the test action
ibmcloud wsk action invoke /whisk.system/utils/echo -p message hello --result
so that your~/.wskprops
is pointing to the right account. -
Whisk Deploy (wskdeploy) is a utility to help you describe and deploy any part of the OpenWhisk programming model using a Manifest file written in YAML. You'll use it to deploy all the Cloud Function resources using a single command. You can download it from the releases page and select the appropriate file for your system.
Clone the fb-watson
locally. In a terminal, run:
$ git clone https://github.com/libardolara/fb-watson
Create a Watson Assistant instance.
- Copy the username and password in the Credentials section and paste them in the
params.json
file in the valueswa_username
andwa_password
- If the service uses IAM API Key authentication, then copy the API Key in the Credentials section and paste it in the
params.json
file in the value ofwa_api_key
- Click the launch tool button on the service main page
- Create a new Workspace in your preferred language or import the
sample_workspace.json
- After creating and/or developing the assistant open the service credentials, copy the Workspace ID and paste it in the
params.json
file in the valueswa_workspace_id
Create a Watson Visual Recognition instance.
- Copy the API Key in the Credentials section and paste it in the
params.json
file in the value ofvr_api_key
- Click the launch tool button on the service main page
- If its the first time you use Watson Studio, this will prepare the environment creating a Cloud Object Storage.
- Click the Create Model button for a custom classifier
- Give a name to the project
- Make sure that the Storage and Watson Visual Recognition are set
- Click the Create button
- Change the name of the Visual Recognition Model
- Upload the
.zip
files with the positive classes and the negative class
- Click the 3 button menu and then click the Add Model only for the positive classes
- Click the Negative default class
- Drag the negative
.zip
into the center of the screen - Return to the main page of the model and click the Train Model button
- When the trainig is done, go to the model detail view.
- Copy the Model ID and paste it in the
params.json
file in the value ofvr_model_id
- You can test the model in the Test tab, draging a new image to classify
Create a Cloudant instance and choose Use both legacy credentials and IAM
for the Available authentication method option.
- Create credentials for this instance and copy the url in the
params.json
file in the value ofcloudant_url
Choose one of the deployment methods
Click and follow the instructions to deploy using the toolchain.
You can also deploy them directly from the CLI by following the steps in the next section.
This approach deploy the Cloud Functions with one command driven by the runtime-specific manifest file available in this repository.
Make sure you have the right parameter variables in the params.json
file. Deploy the Cloud Functions using wskdeploy
. This uses the manifest.yaml
file in this root directory.
$ wskdeploy
You may want to undeploy them later with
wskdeploy undeploy