Guide using Cloud Tasks for Avada projects

Cloud Task is a GCP product allows us to run scheduled tasks with high scalability, a backoff time in seconds. See Firebase guide Enqueue functions with Cloud Tasks

Introduction

According to Firebase documentation, we have this example:

For example, imagine that you want to create backups of a large set of image files that are currently hosted on an API with a rate limit. In order to be a responsible consumer of that API, you need to respect their rate limits. Plus, this kind of long-running job could be vulnerable to failure due to timeouts and memory limits.

We encounter this as well while developing apps on Firebase. Sometimes, we have to use delay functions to delay handling the rate limits. Or in some other scenarios, we have two simultaneous webhooks that arrive at the same time, we want to delay the execution of one webhook over the other to make sure that the logic is handled in the correct order.

Why not it before?

The Cloud Task has its limit as well. Even though it allows you to delay the execution for a few seconds, or more with flexibility, it has no local Emulator which makes the local development workflow hard. You cannot delay it online to Firebase to test it.

Moreover, this dedicated API supports logging and debugging quite bad, compared to having a Cron Scheduler along with Firestore queueing mechanism. However, we can manage to implement some basic emulator for this.

How to implement

Cloud task service

Create your cloudTaskService.js and insert this content. Change your local port of Functions from 5011 to your project's if required.

import {getFunctions} from 'firebase-admin/functions';
import appConfig from '../config/app';
import fetch from 'node-fetch';
import {delay} from '@avada/utils';

/**
 * @description
 * There is no local emulator for cloud task, so if there is delay, we use nodejs delay to handle on local
 * @link https://firebase.google.com/docs/functions/task-functions?gen=1st
 * @param functionName
 * @param opts
 * @param data
 * @returns {Promise<any>}
 */
export async function enqueueTask({functionName, opts = {}, data = {}}) {
  if (appConfig.isLocal) {
    if (opts.scheduleDelaySeconds) {
      await delay(opts.scheduleDelaySeconds * 1000);
    }

    return fetch(
      `http://localhost:5011/${process.env.GCLOUD_PROJECT}/us-central1/${functionName}`,
      {
        headers: {Accept: 'application/json', 'Content-Type': 'application/json'},
        method: 'POST',
        body: JSON.stringify({data})
      }
    );
  }
  if (!appConfig.isLocal) {
    const queue = getFunctions().taskQueue(functionName);

    const enqueues = [];
    enqueues.push(queue.enqueue(data, opts));
    await Promise.all(enqueues);
  }
}

Trigger the enqueue

For example, you can enqueue the task just like publishing a pubsub event.

/**
 *
 * @link todo: need to include
 * The tier detection is delayed because it may have been detected when earning sign-up points
 */
await enqueueTask({
  functionName: ENQUEUE_SUBSCRIBER_FUNC_NAME,
  opts: {
    scheduleDelaySeconds: 3
  },
  data: {type: 'detectSyncTier', data: {shop, customer: data}}
});

In the index.js add the handler for this enqueue:

// ---------------------- Enqueue handlers ----------------------
export const enqueueSubscriber = functions.tasks.taskQueue().onDispatch(enqueueHandler);

In the handler you can do for example like this:

import {detectSyncTier} from '../../repositories/tierRepository';
import isEqualDate from '../../helpers/datetime/isEqualDate';
import {saveCustomerByShopifyId} from '../../repositories/customerRepository';
import {pick} from '@avada/utils';

export const ENQUEUE_SUBSCRIBER_FUNC_NAME = 'enqueueSubscriber';

/**
 *
 * @param queueData
 * @returns {Promise<void>}
 */
export default async function enqueueHandler(queueData) {
  try {
    const {type, data} = queueData;
    console.log('Running enqueueHandler', type);

    switch (type) {
      case 'detectSyncTier': {
        const {shop, customer} = data;
        console.log(
          'customer info in enqueue handler',
          shop.id,
          JSON.stringify(pick(customer, ['email', 'createdAt', 'updatedAt', 'orderCount']))
        );
        const {customer: customerData} = await detectSyncTier(shop, customer, true);
        /**
         * @description
         * Allow saving updated customer if updated time is more than 3 seconds to created time to prevent duplicate customer sign up
         *
         * @link https://trello.com/c/y0YzUHps
         */
        if (!isEqualDate(customer.createdAt, customer.updatedAt, 3)) {
          await saveCustomerByShopifyId({
            shop,
            data: customer,
            customer: customerData
          });
        }
        break;
      }
      default:
        break;
    }
  } catch (e) {}
}

Notes

The emulators

We dont have an emulator at this moment, so this approach will use the delay function in local environment, in the Firebase, it will work just like a Pubsub.

Saving CPU times

If you have tasks that have CPU-intensive, or wasting your CPU times delaying a few seconds. In sum, if it is a webhook listener, it will waste millions of seconds in CPU time billing. So, consider to use this approach.

Remember to enable the service

In order to use it, you need to enable the Cloud Task API in your project. Besides, assign these two IAM roles: