DatabasePython

BrokerDatabasePython

TOM Toolkit broker to query a BigQuery table via the Python API.

Relies on ConsumerDatabasePython to manage the connections and work with data.

See especially:

BrokerDatabasePython.request_alerts

Query alerts using the user filter and unpack.

class tom_pittgoogle.broker_database_python.BrokerDatabasePython

Pitt-Google broker to query alerts from the database via the Python client.

Base class: tom_alerts.alerts.GenericBroker

fetch_alerts(parameters)

Entry point to query and filter alerts.

form

alias of tom_pittgoogle.broker_database_python.FilterAlertsForm

request_alerts(parameters)

Query alerts using the user filter and unpack.

The SQL statement returned by the Consumer implements the current user filter.

Returns

alerts (List[dict])

to_generic_alert(alert)

Map the Pitt-Google alert to a TOM GenericAlert.

class tom_pittgoogle.broker_database_python.FilterAlertsForm(*args, **kwargs)

Basic form for filtering alerts; currently implemented in the SQL statement.

Fields:

objectId (CharField)

candid (IntegerField)

max_results (IntegerField)

property media

Return all media required to render the widgets on this form.

ConsumerDatabasePython

Consumer class to manage BigQuery connections via Python client, and work with data.

BigQuery Python Client docs: https://googleapis.dev/python/bigquery/latest/index.html

Used by BrokerDatabasePython, but can be called independently.

Basic workflow:

consumer = ConsumerDatabasePython(table_name)

sql_stmnt, job_config = consumer.create_sql_stmnt(parameters)
query_job = consumer.client.query(sql_stmnt, job_config=job_config)

alerts = consumer.unpack_query(query_job)  # List[dict]

See especially:

ConsumerDatabasePython.authenticate	Guide user through authentication; create OAuth2Session for credentials.
ConsumerDatabasePython.create_sql_stmnt	Create the SQL statement and a job config with the user's parameters.
ConsumerDatabasePython.unpack_query	Unpack alerts from query_job; run callback if present.

class tom_pittgoogle.consumer_database_python.ConsumerDatabasePython(table_name)

Consumer class to query alerts from BigQuery, and manipulate them.

Initialization does the following:

• Authenticate the user via OAuth 2.0.

• Create a google.cloud.bigquery.Client object for the user/broker to query database with.

• Check that the table exists and we can connect.

To view logs, visit: https://console.cloud.google.com/logs

• Make sure you are logged in, and your project is selected in the dropdown at the top.

• Click the “Log name” dropdown and select the table name you instantiate this consumer with.

TODO: Give the user a standard logger.

authenticate()

Guide user through authentication; create OAuth2Session for credentials.

The user will need to visit a URL, authenticate themselves, and authorize PittGoogleConsumer to make API calls on their behalf.

The user must have a Google account that is authorized make API calls through the project defined by the GOOGLE_CLOUD_PROJECT variable in the Django settings.py file. Any project can be used, as long as the user has access.

Additional requirement because this is still in dev: The OAuth is restricted to users registered with Pitt-Google, so contact us.

TODO: Integrate this with Django. For now, the user interacts via command line.

create_sql_stmnt(parameters)

Create the SQL statement and a job config with the user’s parameters.

unpack_query(query_job, callback=None, **kwargs)

Unpack alerts from query_job; run callback if present.

A basic filter is implemented directly in the SQL statement produced by create_sql_stmnt. More complex filters could be implemented here via a callback function.