windy10v10ai-cloud

Backend for Windy 10v10ai with Firebase

• Built With
• Get Start
  • Installation
  • Running the app
  • Local end points
• Maintenance
  • Deploy
  • Backup Firestore
  • Update dependencies
• Admin API
  • Need
  • CLI
  • Postman

Built With

• Firebase
  • Hosting
  • Functions
  • Realtime Database
• NextJS

Get Start

Installation

Need

• Java
• Node v20
  • Recommend install node use nvm

# firebase setting
npm install -g firebase-tools
firebase login

# setup package
npm install

# web setting
firebase experiments:enable webframeworks

Set GCP (Optional)

Install gCloud SDK: https://cloud.google.com/sdk/docs/install

# Authenticate with gcloud
gcloud auth login

Running the app

Start Firebase Emulator

npm run start

Start Firebase Emulator with data

Need Authenticate with gcloud

# download data from storage (run only once)
rm -rf firestore-backup
mkdir firestore-backup
(cd firestore-backup && gsutil -m cp -r \
  "gs://windy10v10ai.appspot.com/firestore-backup/20240529/20240529.overall_export_metadata" \
  "gs://windy10v10ai.appspot.com/firestore-backup/20240529/all_namespaces" \
  .)

# start emulator with data
npm run start:with-data

gsutil install && 运行 gcloud init login ad https://cloud.google.com/storage/docs/gsutil_install?hl=zh-cn#deb

Run API

# debug (need start firebase emulator)
(cd api && npm run start:debug)

# REPL
(cd api && npm run start -- --entryFile repl)

# unit tests
(cd api && npm run test)

# e2e tests (need stop firebase emulator)
(cd api && npm run test:e2e)

Tips: If debug or e2e test not working with address already used error, kill nodejs process by pkill -f node, or try to restart winnat

net stop winnat
net start winnat

Local end points

• Firebase Hosting: http://localhost:5000/api/
• Debug end points: http://localhost:3000/api/
• Function (Not used): http://localhost:5001/windy10v10ai/asia-northeast1/client/api/
• Firebase Emulator: http://localhost:4000/
• OpenAPI Document (Swagger): http://localhost:3000/api-doc

Maintenance

Deploy

Deploy with Github Action

Github Action will deploy automatically when push to main branch.

• main: Deploy Firebase Functions and Hosting

Deploy Manually

• Deploy all

firebase deploy

• Deploy part

# Deploy api function only
firebase deploy --only functions:client
firebase deploy --only functions:afdian
firebase deploy --only functions:patreon
firebase deploy --only functions:admin
# Deploy all function
firebase deploy --only functions

# Deploy hosting only
firebase deploy --only hosting
# Deploy function and hosting
firebase deploy --only functions,hosting

Set secret environment variables

1. Create env in secret manager
2. Set function run with secrets in index.ts
3. Use secrets as process.env.SECRET_NAME in code

Allow/Disable unauthenticated HTTP function invocation

https://cloud.google.com/functions/docs/securing/managing-access-iam#allowing_unauthenticated_http_function_invocation

Backup Firestore

https://console.cloud.google.com/firestore/databases/-default-/import-export?project=windy10v10ai

Update dependencies

• Update package.json

# install tool
`npm install -g npm-check-updates`

# cd to dir
cd api
# update package.json
ncu -u
# update package-lock.json
npm update

Use Admin API

Need

• gcloud cli

CLI

curl -H "Authorization: bearer $(gcloud auth print-identity-token)" https://asia-northeast1-windy10v10ai.cloudfunctions.net/admin/api

Postman

## Get token
echo $(gcloud auth print-identity-token)

Import api/swagger-spec.yaml to postman with variable baseUrl : https://asia-northeast1-windy10v10ai.cloudfunctions.net/admin

Extension

Export Firestore to Bigquery

Setup Stream Firestore to BigQuery

firebase ext:install firebase/firestore-bigquery-export

or Edit firebase.json and create extensions/firestore-bigquery-export-xxx.env

  "extensions": {
    "firestore-bigquery-export-xxx": "firebase/[email protected]"
  }

Deploy Firestore to BigQuery

firebase deploy --only extensions

Import existing documents (need gcp auth)

Guides

Setup gcloud default auth

gcloud auth application-default login

Run import command

sh ./extensions/import-firestore-to-bigquery.sh

Generate schema views

Create schema file table_name_schema.json and run command.

How to create schema file

sh ./extensions/generate-schema-views.sh