Tableau Server Log Analytics: Easy Parser

I’ve mentioned before but it’s worth mentioning again: Log Analytics and Tableau Server is a wonderful thing. There’s a ton of helpful information in Tableau logs (and pretty much *all* logs) which, along with the PostgreSQL data, make for a very good data toolbox.

I’ve also mentioned Logentries a lot when digging through Tableau logs. There are many reasons I use the tool, but the one which makes it the most useful is: centralized log analysis. Essentially the workflow goes like this: Tableau –> Logs –> Logentries –> Tableau (and around and around). It’s a positive feedback loop of valuable data which can give you insight into things such as:

  • What workbooks take the longest to load and *where* they are geographically
  • What user downloads the most data, how much and how long does it take
  • Http 404s
  • Filters used by a user/workbook
  • Data sources used by a user/workbook
  • and more!

With Tableau, you’re either leveraging a Log Analytics strategy or you’re not. I cannot stress how vital it is for Tableau Server administrators to at least have some plan in place for when you are suddenly inundated with a ‘slow’ server and/or site.

That said, often times it’s easier to have a few functions and tools to make the ad-hoc or automated analysis easier. Here’s one: we’ll wrap the Logentries REST API in a PowerShell function. This will simply allow us to pull log data from Apache or VizQL based off of a simple parameter.

What’s returned is a neatly formatted csv which you can then import into Tableau, add to a database or simply do some quick research. For example, if you want to ensure excessive 404s are handled, you can simply use this function with a filter, parse, and lookup the offending IP address.  If necessary you’d add those IPs to a firewall rule.

More specifically, here’s an example of how you would use the function in PowerShell:

Get-TsLog -leAcctKeyVizQL 'your VizQL key' -leFilterVizQL 'k=end-update-sheet' -workpathVizQL "C:\users\$env:username\Documents" -apikey 'your Logentries API key'
vapor_rub_0
Here’s where your log data (parsed) can become a great means to improve performance and react before things happen

The added benefit of adding this type of aggregated data to your own Tableau data model and database is that it gives the admin some data for historical purposes, planning and learning.

So, here’s the module on the PowerShell gallery. Let me know if there are questions.

 

Slack your Tableau Extract – Part II

Ever wish you could drop the name of your Tableau workbook and/or data source into a Slack channel and have it automatically refresh? What if you’re developing and need to get a quick refresh completed? What if you don’t have tabcmd installed on your machine? What if you want to add step at the end of your pipeline that drops the name of the content into the Slack channel?

ts-refresh-extract-0
Example of name of Tableau content

I’ve talked about this before but what had to happen was the extract needed to exist in the ‘background_jobs’ table. Well, that won’t always happen as people will be doing this for the first time. So, we needed to expand it a bit to include *all* possibilities (workbook and data sources). Also, in this much improved version, we Slack back to the user and let them know their extract is completed.

ts-refresh-extract-2
Process for *each* extract (all dynamic) 

That’s the beauty of the ‘Tableau-Slack-Logentries‘ integration. When you have a decent amount of parts, the whole becomes a fascinating thing.

Here are the steps:

  • get the data from the Logentries webhook
  • Process the data for each extract
  • getting current slack users: don’t need to do this often (unless you want to)
  • getting valid list of workbooks and data sources
  • Processing list of extracts : rolling 24 hours
  • getting valid list of workbook / data source owners
  • create Slack content object: basically it must add up to a certain number to run (for example, if the person who dropped the name in the channel isn’t the owner, it won’t succeed).
  • Log it!

 

ts-refresh-extract-3
Tableau Server example of completed extract. 
ts-refresh-extract-1
Corresponding message back from Tableau Server

If anyone is interested in the code and / or a demo, please let me know and I’ll be happy to show it.

Automatically remove (and archive) extracts that fail more than ‘n’ times

Keep it clean

Every now and then, Tableau extracts aren’t refreshed properly. That, in and of itself, isn’t a bad thing. What we’re worried about are the extracts that continue to fail day after day. They consume resources and, most importantly, don’t give a true picture of the data.

ts-extract-fails-0
Continual extract failures

Here’s a simple script that queries postgres and grabs the extracts which have failed n times (you can choose your threshold). At a very high level, it does the following:

  • Get list of failed extracts (most recent fail date should be the current date)
  • Limit to those only above your desired threshold
  • Use REST API to pull extracts from Tableau Server
  • Do a diff on list and downloaded files and remove only those which are equal
  • Archive twb/twbx/tds/tdsx so users can refer to this later
  • Delete content from Tableau Server
ts-extract-fails-1
Here is how we dynamically switch from site to site *and* from content resource (data source or workbook) with the REST API

Taking it a step further

If you have a Log Analytics strategy in place, you can send the output of the script to a file and have the log agent follow it. This will give you and the rest of the Tableau admin team some insight to what is failing (other than what I’ve talked about before).

You can also integrate with Slack in order to notify the user/owner that his/her workbook will be remove until the failure is fixed.