PAXminer

This page provides information on the F3 PAXminer tool for capturing and parsing beatdown statistics and attendance records from Backblasts.

Wouldn’t it be awesome if your PAX could get summaries of their beatdown attendance like this spoon fed to them – and it was fully automated?

PAXminer is currently running in a multi-region, cloud based configuration. Adding a new region is easy.

The technical details (that you don’t really need to know):

PAXminer runs on a cloud-based Amazon AWS EC2 server. Each region defines which Slack channels (within their respective Slack workspace) it uses to post Backblasts to. Region by region, PAXminer pulls all Backblast messages from those Slack Channel using the Slack API and then parses through each Backblast to find the key statistics and attendees. Those stats are then stored in an AWS RDS MySql database.

Each region using PAXminer has it’s own schema within the PAXminer database. While the creation of stats and graphs sent to the region’s Slack channels is automated, each region also has direct access to the database for ad-hoc use and data quality management. It’s easy, you don’t need to be a database expert to use this.

A GitHub repo exists containing all of the underlying scripts that make up PAXminer, accessible here:

https://github.com/srschaecher/PAXminer

While you are welcome to download the code and configure you’re own environment, you are encouraged to utilize the existing multi-region environment. Contact @Beaker for more information.

Already running PAXminer? Here are some helpful how-to documents:

And wouldn’t it be great if you could see posting stats for all HIMs across an AO – or even across a region? Think of the posting competitions that could be had…

Or maybe you’d like to get some healthy competition across AOs to see which one can get the most PAX to attend throughout the course of a month?

“Yeah, but we like having our BackBlasts on our website…”

Well – would you like them to be easily searchable? Maybe you want to see every BackBlast from your favorite Q – Ringer? Or maybe you want some specific ideas from all backlists from a particular AO?

Oh that reminds me. It sure would be nice to have real time stats on our website too.

Well, guess what? Now – you can.

PAXminer is a Backblast analysis pipeline that retrieves and parses Backblasts from an F3 unit’s Slack channels in order to pull key information out and store that information in a database for recordkeeping purposes. It also automatically generates statistical graphs and charts and sends those to individual PAX, AOs and Regions (1st-F) to highlight monthly or yearly posting statistics. Additionally, having the data in a database enables easy creation of real-time graphs and stats directly on your F3 Region website.

PAXminer pulls the following infomation from Backblasts and sends it to a shared database in AWS RDS (Amazon Web Services – Relational Database Service). Each region using the shared PAXminer environment gets a dedicated schema for their data.

AO
Date of the Beatdown
Who was the Q
Was there a Co-Q?
What PAX attended?
How many attended in total?
Any FNGs?
The entire BackBlast text is also stored for easily serving it to the website and making it searchable.

This tool started out with me saying

“there HAS to be a better way to do this”

as we were collecting our F3 posting stats manually in Excel – so now, there is. We are adding new things to the process every day! The more regions that get involved, the better and more standardized we can make it! All collaboration is welcome!

Tracking this information lets you ask many different questions about your F3 attendance patterns, such as:

How many times has Beaker (that’s me) attended a beatdown this week/month/year?
What AO does Beaker most often attend?
What days of the week are busiest across all AOs?
Is attendance high or low when a specific person Qs?
How many times did GMO (the F3STL Naantan) Q’d last month?
Who are our Cotters that we need to reach out to? PAX that have been MIA for a while?
Is it time to open a new AO?
and the list goes on…

The PAXminer philosophy is to minimize the number of places Q’s have to post the Beatdown information to just 1 – Slack. If you’re posting backblasts to WordPress and to Slack (or your communication app, whatever that may be), then you’re not making it easy on your Q’s. Additionally, having the backblasts in Slack is imperative as we absolutely want to encourage the mumblechatter that comes along with a backblast… so we’ve landed on Slack being the place where everyone posts BBs.

PAXminer pulls the backblast automatically from Slack (using the Slack API) and parses it for all of the statistics, sending those to the database. It also stores the entire BB text as well so we can then serve it back to the WordPress website from the database. This path keeps it simple. Nearly 100% of our PAX use Slack, but the minute we ask people to use both Slack AND the website, adoption drops. By keeping the BBs posted through slack and then automating the Slack -> Website transfer of the BB, it makes the BBs searchable in the website as well. PAXminer uses a flexible backblast format that all Q’s use to post BBs. They are posted directly to AO slack channels. It was important to make the process as simple as possible for Q’s. How it works:

Q’s post their BB to slack
An automated process runs in the background that pulls all BBs from Slack and parses them, collecting the statistics
It identifies the Q, the AO, the date, all PAX in attendance, and FNGs
It puts all of that data into a database running in the cloud

Additionally, PAXminer also stores the full BB text that can easily be served back to the website. Using simple WordPress plugins, a website can show a filterable and searchable list of all beatdowns and clicking on one can expand to show the full BackBlast. What’s even cooler, though, is spoon-feeding the stats back to the PAX. With all of the stats in the database, an automated process runs at various times that sends lots of great statistics back to the PAX, either in direct slack messages to each PAX (see above), summary stats by AO to each AO channel in Slack, and region-wide stats to the region’s 1st-F channel. Each PAX will get an individualzed posting graph at the end of each month like the one shown above. It has proven to be extremely motivating for people. The process also creates graphs for total posting statistics and sends to each AO slack channel so people can see how many folks are posting and who does all the Q’ing (which has been motivating to start getting more people to Q). PAXminer is designed so adding a new region is very easy. The biggest requirement/change for any region to start using this is simply to adopt the BB format and to post them to Slack. The BackBlast template is fairly flexible; however, it must start with the word “Backblast” (or “Back Blast”). This is how PAXminer knows to pull it out and start pulling out the stats.

Backblast! (Give your workout a creative name, make sure the first word of the message is Backblast with NO SPACES before it)
Date: Day, MM/DD/YY (note – the format is pretty flexible, you can use 05/05/2021 or Wednesday, May 5, 2021. If entering on your phone, make sure your phone doesn’t turn this into a telephone# hyperlink like seems to happen sometimes… if the date turns blue, your phone made it a hyperlink and it will break)
AO: AO Name (use the channel name if possible like #ao_wet_willy)
PAX: Tag every PAX in attendance with @NAME. Make sure they’re tagged/blue. If not on Slack, just list their F3 name.
FNGs: Say the total # of FNGs, then list their new F3 names here separated by commas. They mostly won’t have Slack accounts set up yet. Even if they do, list them here.
Q: tag Q and co-Q if applicable with @NAME. Q first, then Co-Q second.
Total: Total number including Q, PAX, and FNGs. Number only, no text.
Conditions: How was the morning? Get colorful.

Warm-Up:Exercise 1
· Exercise 2
· …

The Thang:
– Describe your BD.

Mary:
·      Exercise 1
·      Exercise 2
·      …
Announcements:
·      Announcement 1
·      Announcement 2
·      …

Count-o-Rama
Name-O-Rama

Q took us out in… <prayer? poem? meditation? Silence? His why? …>

Once your region is set up to start running PAXminer, you will specify which Slack channels that it should look in for BackBlasts. Post them using the above template. As soon as a BackBlast is imported, the Q will receive a confirmation message in Slack like this:

Once your region’s database begins to collect data, not only will your region be able to get the automated graphs and charts above, but you will also get access to the database to run your own custom queries as you’d like. For example, you can pull out some great region statistics with simple SQL queries like that below. Don’t know SQL? Don’t fret – there are some HIMs around that are more than happy to help you out. It’s really not hard.

Ready to get rolling? It’s easy. Click HERE to get started!

Facebook Twitter Youtube Instagram Envelope

F3 ST LOUIS

Join us in The Gloom

FIND A WORK OUT

F3 Annapolis