Pitt-Google integration with TOM Toolkit

This repo contains 3 proof-of-concept implementations of a TOM Toolkit GenericBroker class which fetch alerts from Pitt-Google.

Contact Troy Raen with questions or for authentication access (Slack @troyraen, or troy.raen@pitt.edu).

Basic Overview

TOM Broker 3 ways

Implementation

Connects to

Via

Comments

StreamRest

Pub/Sub streams

REST API

Closest to “standard” implementation using HTTP requests. Uses batch-style message pulls.

StreamPython

Pub/Sub streams

Python client

Recommended for listening to a full night’s stream. Uses a streaming pull in a background thread.

DatabasePython

BigQuery database

Python client

Each implementation relies on 2 classes, a Broker and a Consumer:

2 classes for each implementation

Broker

Consumer

  • Fetches alerts from Pitt-Google using a Consumer

  • Handles the stream/database connections and unpacks the returned data.

  • Base class: tom_alerts.alerts.GenericBroker

  • Python methods use Google’s client APIs (Pub/Sub, BigQuery)

  • REST method uses a requests_oauthlib.OAuth2Session object for HTTP requests

Here we use Broker and Consumer generically to refer to any of the specific implementations, which have names like BrokerStreamRest.