Month: May 2023

Posted on

Project Debrief: Filter Feed

Note: this post is probably of low interest. I am simply making a record of some little personal projects I have made.

Project Goals

I subscribe to several podcasts and RSS feeds which include some posts I’m not interested in. For example, USGS has a feed of volcano statuses but I only want to subscribe to my local volcano. My podcast app and feed reader don’t have filtering of episodes built in, so I wanted to create an app that would apply filters while proxying requests.

High Level Design

Filter feed is a Google Cloud Run app. Each feed has a unique ID, and and a app URL containing that ID is given to the feed reader. When the feed reader loads the feed from the app, configuration on what filtering to do is loaded from a Datastore entry, the upstream feed is fetched and filtered before being returned to the feed reader app.

Implementation Notes

Code: https://github.com/davidn/filter-feed

We use jQuery Query Builder as an interface for defining power filters on feeds:

Filter feed admin illustrating Query Builder for specifying a custom filter

Much of the code is dedicated to correctly handling both Atom and RSS, as well as coaxing xml.etree to make the deserialize-filter-serialize round trip preserve as much of the original XML as possible.

An admin interface was a later addition. This necessitated adding user and login to prevent 3rd parties modifying my filters. For this I used flask-security-too. I would note that was not a polished experience, and I would recommend against Flask for any project that requires log-ins and permissions.

Project Fate

I continue to use this app for RSS feed filtering, although I no longer use it for podcasts as I found a podcast app with enough filtering built in. If this app would be useful to you, I can provide you an account, just reach out to me.

Posted on

Project Debrief: Dice Calculator

Note: this post is probably of low interest. I am simply making a record of some little personal projects I have made.

Project Goals

My personal motivation for this project was to learn about voice assistants. The actual project goal, however, was to produce a dice calculation voice assistant feature that understood the ins-and-outs of D&D 5e. I wanted to be able to say “with advantage”, apply modifiers, and even reference spells.

Design

Dice Calculator was a “Conversational Action” on Google Assistant, which meant you could say “Hey Google, talk to Dice Calculator” to invoke it. A single intent dispatched all requests via Dialogflow to a app running in Google Cloud Run.

This app parsed the request using a grammar that was part manually specified and part generated from a database of D&D 5e spells and weapons. The parsed tree then went through a series of transformations, for example transforming advantage (a D&D concept) into the max of two copies of whatever was being done with advantage. The final transformation actually rolls dice and does arithmetic, resulting in the final answer.

This final answer was then composed into a dialog response, and sent back to the user.

Implementation Notes

The code is on github at https://github.com/davidn/dice-calculator

Dialogflow webhook fulfillment is documented as a JSON API. However, knowing that Google uses protobuf internally and wanting to use structured serialization/deserialization, I found and used the protobuf definitions with protobuf json_format. This may have been a mistake, as the protobuf library is large and resulted in slow cold-starts in Google Cloud Run, causing a long tail latency.

For debugging, the code is instrumented with Google Cloud Trace & Logs allowing easily seeing the inner workings of each request:

The trace of a request shows the timing of all the processing steps, hierarchically.

For monitoring, I used a Google Cloud Monitoring uptime check which invokes a Google Cloud Function endpoint which in turn makes a full request to the Cloud Functions app, and checks the result. This is needed because a simple uptime check directly to the app would not contain a valid request nor check the validity of the response.

Fate

Sadly Google is shutting down the ability for developers to provide custom conversation agents with Google Assistant, so Dice Calculator is no longer operational.