-
Notifications
You must be signed in to change notification settings - Fork 2k
Cloud Task + GAE + Functions Tutorial #1488
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from 7 commits
aad4ff6
5b0ef75
5ce9b69
9e7d19e
6c3fedc
74a300d
a90062e
1d22519
9bb9e1f
2c2187b
6477d03
acf4a1e
73d7ab3
95e7ac4
f0eb573
b5e103a
176961b
cadeddd
efee1ab
cfbc82b
fe94dbd
4202798
be4588e
3446705
a0b74b1
4e8f7ea
5400912
3d0f6af
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,7 @@ | ||
| # Format: //devtools/kokoro/config/proto/build.proto | ||
|
|
||
| # Set the folder in which the tests are run | ||
| env_vars: { | ||
| key: "PROJECT" | ||
| value: "cloud-tasks/app" | ||
| } |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,7 @@ | ||
| # Format: //devtools/kokoro/config/proto/build.proto | ||
|
|
||
| # Set the folder in which the tests are run | ||
| env_vars: { | ||
| key: "PROJECT" | ||
| value: "cloud-tasks/function" | ||
| } |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,22 @@ | ||
| # Format: //devtools/kokoro/config/proto/build.proto | ||
|
|
||
| # Download trampoline resources. These will be in ${KOKORO_GFILE_DIR} | ||
| gfile_resources: "/bigstore/cloud-devrel-kokoro-resources/trampoline" | ||
|
|
||
| # Download secrets from Cloud Storage. | ||
| gfile_resources: "/bigstore/cloud-devrel-kokoro-resources/nodejs-docs-samples" | ||
|
|
||
| # All builds use the trampoline script to run in docker. | ||
| build_file: "nodejs-docs-samples/.kokoro/trampoline.sh" | ||
|
|
||
| # Tell the trampoline which build file to use. | ||
| env_vars: { | ||
| key: "TRAMPOLINE_BUILD_FILE" | ||
| value: "github/nodejs-docs-samples/.kokoro/build.sh" | ||
| } | ||
|
|
||
| # Configure the docker image for kokoro-trampoline. | ||
| env_vars: { | ||
| key: "TRAMPOLINE_IMAGE" | ||
| value: "gcr.io/cloud-devrel-kokoro-resources/node:10-user" | ||
| } | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,82 @@ | ||
| # Node.js Cloud Tasks sample for Google App Engine and Cloud Functions | ||
|
averikitsch marked this conversation as resolved.
Outdated
averikitsch marked this conversation as resolved.
Outdated
|
||
|
|
||
| This sample shows how to create [Cloud Tasks](cloud-tasks) on | ||
| [Google App Engine Standard](gae-std) to trigger a [Cloud Function](cloud-func) | ||
|
averikitsch marked this conversation as resolved.
Outdated
|
||
| in order to send an email postcard. | ||
|
averikitsch marked this conversation as resolved.
Outdated
averikitsch marked this conversation as resolved.
Outdated
|
||
|
|
||
| * The App Engine application calls the Cloud Tasks API to add a scheduled task to | ||
|
averikitsch marked this conversation as resolved.
Outdated
|
||
| the queue. | ||
|
|
||
| * The queue processes tasks and sends requests to a Cloud Function. | ||
|
|
||
| * The Cloud Function calls the SendGrid API to send an email postcard. | ||
|
averikitsch marked this conversation as resolved.
Outdated
|
||
|
|
||
| [cloud-tasks]: https://cloud.google.com/tasks/docs/ | ||
| [gae-std]: https://cloud.google.com/appengine/docs/standard/nodejs/ | ||
| [cloud-func]: https://cloud.google.com/functions/ | ||
|
|
||
| ## Setup | ||
|
|
||
| Before you can deploy the sample, you will need to do the following: | ||
|
averikitsch marked this conversation as resolved.
Outdated
|
||
|
|
||
| 1. Enable the Cloud Function and Cloud Tasks APIs in the | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggestion
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Can I say, for example, "The Cloud Function does ..." or should I just say "The function does ..."? |
||
| [Google Cloud Platform Console](https://console.cloud.google.com/apis/library). | ||
|
|
||
| 1. Install and initialize the [Cloud SDK](https://cloud.google.com/sdk/docs/). | ||
|
|
||
| 1. Create a [SendGrid account](https://sendgrid.com/free) and generate a | ||
|
averikitsch marked this conversation as resolved.
Outdated
|
||
| [SendGrid API key](https://app.sendgrid.com/settings/api_keys). | ||
|
|
||
| 1. Create a Cloud Tasks queue: | ||
|
|
||
| ``` | ||
| gcloud tasks queues create my-queue | ||
| ``` | ||
|
|
||
| 1. Create a service account in the | ||
|
averikitsch marked this conversation as resolved.
Outdated
|
||
| [IAM & Admin UI](https://console.cloud.google.com/iam-admin/iam) with the role | ||
| Cloud Functions Invoker. | ||
|
averikitsch marked this conversation as resolved.
Outdated
|
||
|
|
||
| ## Deploying to Cloud Function | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Cloud Functions is plural.
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. done. |
||
|
|
||
| 1. To deploy the function with an HTTP trigger, run the following command in | ||
|
averikitsch marked this conversation as resolved.
Outdated
|
||
| the `function/` directory replacing `<sendgrid_api_key>` with your key: | ||
| ``` | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Be sure to indent appropriately.
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. ack. |
||
| gcloud functions deploy sendPostcard --runtime nodejs8 --trigger-http --set-env-vars SENDGRID_API_KEY=<sendgrid_api_key> | ||
| ``` | ||
|
|
||
| 1. Restrict requests to your function to only authenticated users: | ||
|
|
||
| * Select your function in the [Cloud Function UI](https://console.cloud.google.com/functions/list) | ||
|
averikitsch marked this conversation as resolved.
Outdated
|
||
| and click “SHOW INFO PANEL” in the upper right hand corner. | ||
|
averikitsch marked this conversation as resolved.
Outdated
|
||
|
|
||
| * Under Cloud Functions Invoker header, delete the member `allUsers`. | ||
|
averikitsch marked this conversation as resolved.
Outdated
|
||
|
|
||
| * Click “Add members”. Add `allAuthenticatedUsers` with the role | ||
|
averikitsch marked this conversation as resolved.
Outdated
|
||
| `Cloud Function Invoker`. | ||
|
|
||
| ## Deploying to Google App Engine | ||
|
|
||
| 1. In the `app/` directory, update the values in `app.yaml`: | ||
| ``` | ||
| env_variables: | ||
| QUEUE_NAME: "my-queue" | ||
| QUEUE_LOCATION: "us-central1" | ||
| FUNCTION_URL: "https://<region>-<project_id>.cloudfunctions.net/sendPostcard" | ||
| SERVICE_ACCOUNT_EMAIL: "<member>@<project>.iam.gserviceaccount.com" | ||
| ``` | ||
| To find these values, use the following commands: | ||
|
averikitsch marked this conversation as resolved.
Outdated
|
||
| ``` | ||
| gcloud tasks queues describe my-queue | ||
| gcloud functions describe sendPostcard | ||
| ``` | ||
|
|
||
| 1. Deploy the app to App Engine standard environment: | ||
|
averikitsch marked this conversation as resolved.
Outdated
|
||
| ``` | ||
| gcloud app deploy | ||
| ``` | ||
|
|
||
| 1. Open the app to send a postcard: | ||
| ``` | ||
| gcloud app browse | ||
| ``` | ||
|
averikitsch marked this conversation as resolved.
Outdated
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,23 @@ | ||
| # Copyright 2019 Google LLC | ||
|
averikitsch marked this conversation as resolved.
Outdated
|
||
| # Licensed under the Apache License, Version 2.0 (the "License"); | ||
| # you may not use this file except in compliance with the License. | ||
| # You may obtain a copy of the License at | ||
| # http://www.apache.org/licenses/LICENSE-2.0 | ||
| # Unless required by applicable law or agreed to in writing, software | ||
| # distributed under the License is distributed on an "AS IS" BASIS, | ||
| # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
| # See the License for the specific language governing permissions and | ||
| # limitations under the License. | ||
|
|
||
| runtime: nodejs10 | ||
|
|
||
| env_variables: | ||
| QUEUE_NAME: "my-queue" | ||
| QUEUE_LOCATION: "us-central1" | ||
| FUNCTION_URL: "https://<region>-<project_id>.cloudfunctions.net/sendPostcard" | ||
| SERVICE_ACCOUNT_EMAIL: "<member>@<project_id>.iam.gserviceaccount.com" | ||
|
|
||
| handlers: | ||
| - url: / | ||
| static_files: index.html | ||
| upload: index.html | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,86 @@ | ||
| /** | ||
| * Copyright 2019 Google LLC | ||
| * Licensed under the Apache License, Version 2.0 (the "License"); | ||
| * you may not use this file except in compliance with the License. | ||
| * You may obtain a copy of the License at | ||
| * | ||
| * http://www.apache.org/licenses/LICENSE-2.0 | ||
| * | ||
| * Unless required by applicable law or agreed to in writing, software | ||
| * distributed under the License is distributed on an "AS IS" BASIS, | ||
| * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
| * See the License for the specific language governing permissions and | ||
| * limitations under the License. | ||
| */ | ||
|
|
||
| 'use strict'; | ||
|
|
||
| async function createHttpTaskWithToken( | ||
| project = 'my-project-id', // Your GCP Project id | ||
| queue = 'my-queue', // Name of your Queue | ||
| location = 'us-central1', // The GCP region of your queue | ||
| url = 'https://example.com/taskhandler', // The full url path that the request will be sent to | ||
| email = '<member>@<project-id>.iam.gserviceaccount.com', // Cloud IAM service account | ||
| payload = 'Hello, World!', // The task HTTP request body | ||
| date = Date.now() // Date to schedule task | ||
|
averikitsch marked this conversation as resolved.
Outdated
averikitsch marked this conversation as resolved.
Outdated
|
||
| ) { | ||
| // Imports the Google Cloud Tasks library. | ||
| const {v2beta3} = require('@google-cloud/tasks'); | ||
|
|
||
| // Instantiates a client. | ||
| const client = new v2beta3.CloudTasksClient(); | ||
|
|
||
| // Construct the fully qualified queue name. | ||
| const parent = client.queuePath(project, location, queue); | ||
|
|
||
| // Convert message to buffer | ||
| const converted_payload = JSON.stringify(payload); | ||
| const body = Buffer.from(converted_payload).toString('base64'); | ||
|
|
||
| const task = { | ||
| httpRequest: { | ||
| httpMethod: 'POST', | ||
| url, | ||
| oidcToken: { | ||
| serviceAccountEmail: email, | ||
| }, | ||
| headers: { | ||
| 'Content-Type': 'application/json', | ||
| }, | ||
| body, | ||
| }, | ||
| }; | ||
|
|
||
| // Calculate time to send message | ||
| const converted_date = new Date(date); | ||
|
averikitsch marked this conversation as resolved.
Outdated
|
||
| const current_date = new Date(Date.now()); | ||
|
averikitsch marked this conversation as resolved.
Outdated
|
||
|
|
||
| if (converted_date < current_date) { | ||
| console.error('Scheduled date in the past.'); | ||
| } else if (converted_date > current_date) { | ||
| const seconds_diff = (converted_date - current_date) / 1000; | ||
|
averikitsch marked this conversation as resolved.
Outdated
|
||
| // 30 day maximum for schedule time | ||
|
averikitsch marked this conversation as resolved.
Outdated
|
||
| const add_seconds = | ||
|
averikitsch marked this conversation as resolved.
Outdated
|
||
| Math.min(seconds_diff, 30 * 60 * 60 * 24) + Date.now() / 1000; | ||
|
averikitsch marked this conversation as resolved.
Outdated
|
||
| // Add schedule time if after current date | ||
| task.scheduleTime = { | ||
| seconds: add_seconds, | ||
| }; | ||
| } | ||
|
|
||
| const request = { | ||
| parent: parent, | ||
| task: task, | ||
| }; | ||
|
|
||
| // Send create task request. | ||
| try { | ||
| const [response] = await client.createTask(request); | ||
|
averikitsch marked this conversation as resolved.
Outdated
|
||
| console.log(`Created task ${response.name}`); | ||
| return response.name; | ||
| } catch (error) { | ||
| console.error(error.message); | ||
|
averikitsch marked this conversation as resolved.
Outdated
|
||
| } | ||
| } | ||
|
|
||
| module.exports = createHttpTaskWithToken; | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,97 @@ | ||
| <!DOCTYPE html> | ||
| <!-- Copyright 2019 Google LLC | ||
| Licensed under the Apache License, Version 2.0 (the "License"); | ||
| you may not use this file except in compliance with the License. | ||
| You may obtain a copy of the License at | ||
| http://www.apache.org/licenses/LICENSE-2.0 | ||
| Unless required by applicable law or agreed to in writing, software | ||
| distributed under the License is distributed on an "AS IS" BASIS, | ||
| WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
| See the License for the specific language governing permissions and | ||
| limitations under the License. --> | ||
| <html lang="en"> | ||
| <head> | ||
| <meta charset="UTF-8"> | ||
| <title>Send a Postcard</title> | ||
| <link href="https://unpkg.com/material-components-web@latest/dist/material-components-web.min.css" rel="stylesheet"> | ||
| <link rel="stylesheet" href="https://fonts.googleapis.com/icon?family=Material+Icons"> | ||
| <style> | ||
| body { | ||
| text-align: center; | ||
| } | ||
| .mdc-card { | ||
| width: 800px; | ||
| } | ||
| .center .mdc-card { | ||
| margin: auto; | ||
| text-align: center; | ||
| } | ||
| .mdc-text-field { | ||
| margin: 1em; | ||
| } | ||
| </style> | ||
| </head> | ||
| <body class="mdc-typography"> | ||
|
|
||
| <div class="center"> | ||
| <h1 class="mdc-typography--headline2"> Send a E-Postcard! </h1> | ||
| <div class="mdc-card mdc-card--outlined"> | ||
| <!-- Postcard form --> | ||
| <form action="/send-email" method="post"> | ||
| <div class="mdc-layout-grid"> | ||
| <div class="mdc-layout-grid__inner"> | ||
| <!-- Message textarea --> | ||
| <div class="mdc-layout-grid__cell mdc-layout-grid__cell--span-6"> | ||
| <h3>Your message:</h3> | ||
| <div class="mdc-text-field mdc-text-field--textarea mdc-text-field--no-label"> | ||
| <textarea id="textarea" class="mdc-text-field__input" | ||
| rows="8" cols="40" name="message" aria-label="Label"></textarea> | ||
| <div class="mdc-notched-outline"> | ||
| <div class="mdc-notched-outline__leading"></div> | ||
| <div class="mdc-notched-outline__trailing"></div> | ||
| </div> | ||
| </div> | ||
| </div> | ||
| <!-- Sender/Receiver Emails --> | ||
| <div class=" mdc-layout-grid__cell mdc-layout-grid__cell--span-6"> | ||
| <div class="mdc-text-field mdc-text-field--no-label"> | ||
| <input type="text" class="mdc-text-field__input" name="to_email" | ||
| placeholder="Recipient Email" aria-label="Label" required> | ||
| </div> | ||
|
|
||
| <div class="mdc-text-field mdc-text-field--no-label"> | ||
| <input type="text" class="mdc-text-field__input" name="from_email" | ||
| placeholder="Your Email" aria-label="Label" required> | ||
| </div> | ||
| <div> | ||
| <h3>Schedule your postcard:</h3> | ||
| <input id="date-input" type="date" name="date"> | ||
| </div> | ||
| </div> | ||
| </div> | ||
| </div> | ||
| <div class="mdc-card__actions"> | ||
| <div class="mdc-card__action-buttons"> | ||
| <button class="mdc-button mdc-card__action mdc-card__action--button"> | ||
| <span class="mdc-button__label">Send</span> | ||
| <i class="material-icons mdc-button__icon" aria-hidden="true">send</i> | ||
| </button> | ||
| </div> | ||
| </div> | ||
| </form> | ||
|
|
||
| </div> | ||
| </div> | ||
| <script> | ||
| let now = new Date(); | ||
|
averikitsch marked this conversation as resolved.
Outdated
|
||
|
|
||
| let maxDate = new Date(now); | ||
| maxDate.setDate(now.getDate() + 30) | ||
|
averikitsch marked this conversation as resolved.
Outdated
|
||
|
|
||
| document.getElementById('date-input') | ||
| .setAttribute('min', now.toISOString().substring(0,10)) | ||
|
averikitsch marked this conversation as resolved.
Outdated
|
||
| document.getElementById('date-input') | ||
| .setAttribute('max', maxDate.toISOString().substring(0,10)); | ||
| </script> | ||
| </body> | ||
| </html> | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,63 @@ | ||
| /** | ||
| * Copyright 2019 Google LLC | ||
| * Licensed under the Apache License, Version 2.0 (the "License"); | ||
| * you may not use this file except in compliance with the License. | ||
| * You may obtain a copy of the License at | ||
| * | ||
| * http://www.apache.org/licenses/LICENSE-2.0 | ||
| * | ||
| * Unless required by applicable law or agreed to in writing, software | ||
| * distributed under the License is distributed on an "AS IS" BASIS, | ||
| * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
| * See the License for the specific language governing permissions and | ||
| * limitations under the License. | ||
| */ | ||
|
|
||
| 'use strict'; | ||
|
|
||
| const createHttpTaskWithToken = require('./createTask'); | ||
| const express = require('express'); | ||
|
|
||
| const app = express(); | ||
|
|
||
| const QUEUE_NAME = process.env.QUEUE_NAME || 'my-queue'; | ||
| const QUEUE_LOCATION = process.env.QUEUE_LOCATION || 'us-central1'; | ||
| const FUNCTION_URL = | ||
| process.env.FUNCTION_URL || | ||
| 'https://<region>-<project_id>.cloudfunctions.net/sendPostcard'; | ||
|
averikitsch marked this conversation as resolved.
Outdated
|
||
| const SERVICE_ACCOUNT_EMAIL = | ||
| process.env.SERVICE_ACCOUNT_EMAIL || | ||
| '<member>@<project_id>.iam.gserviceaccount.com'; | ||
|
|
||
| app.use(express.urlencoded()); | ||
|
|
||
| app.post('/send-email', (req, res) => { | ||
| // Construct Task payload | ||
| const payload = { | ||
|
averikitsch marked this conversation as resolved.
Outdated
|
||
| to_email: req.body.to_email, | ||
| from_email: req.body.from_email, | ||
| message: req.body.message, | ||
|
averikitsch marked this conversation as resolved.
Outdated
|
||
| }; | ||
|
|
||
| createHttpTaskWithToken( | ||
| process.env.GOOGLE_CLOUD_PROJECT, | ||
| QUEUE_NAME, | ||
| QUEUE_LOCATION, | ||
| FUNCTION_URL, | ||
| SERVICE_ACCOUNT_EMAIL, | ||
| payload, | ||
| req.body.date | ||
| ); | ||
|
|
||
| res.status(202).send('📫 Your postcard is in the mail! 💌'); | ||
| }); | ||
|
|
||
| app.get('*', (req, res) => { | ||
| res.send('OK').end(); | ||
| }); | ||
|
averikitsch marked this conversation as resolved.
Outdated
|
||
|
|
||
| const PORT = process.env.PORT || 8080; | ||
| app.listen(process.env.PORT || 8080, () => { | ||
|
averikitsch marked this conversation as resolved.
Outdated
averikitsch marked this conversation as resolved.
Outdated
|
||
| console.log(`App listening on port ${PORT}`); | ||
| console.log('Press Ctrl+C to quit.'); | ||
| }); | ||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why does cloud-tasks need a customized common.cfg? So far most products don't need one.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Every folder in
.kokorohas acommon.cfgfile, but you are right it is a duplication of the parents.