Even though I consider myself knowledgeable about the multiple GCP products related to data engineering, I had not heard of a use case for Cloud Tasks before.

This post aims to shed some light on the use case for Cloud Tasks by bringing a specific problem to the table and discussing it from a PubSub vs Cloud Tasks perspective.

The Challenge

As part of a customer-data-segmentation project I encountered the challenge of sending user data to the Google Ads Remarketing Audience API, which has the following restrictions:

  1. Every request can only contain around 50.000 user records. …


This guide is intended to be a handy reference for myself when i’m looking for a specific link or command, or setting up a new workstation. I’m expecting this guide to grow with more commands over time.

I’ll be covering the following areas in this guide:

  • Homebrew
  • Pyenv
  • Virtual env
  • Google Cloud Python Client
  • Google Auth Library
  • Authentication from environment variables
  • Handy ~/.bashrc or ~/.zshrc lines

Homebrew, Pyenv & Virtual env

This holy trinity should be taught in all tutorials. If you want control and understanding of your local environment, the combination of homebrew, pyenv & venv is the only way to go.

Homebrew

Homebrew is…

Max Krog

Data Engineer @ Valtech — Stockholm, Sweden

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store