Use the Logs: Tableau Server Log Analytics – Part 4

Week 4: Data Engine

For a review of what we’ve covered, make sure to check out the previous three weeks of log analytics for Tableau Server:

Week 1: Backgrounder

Week 2: Apache

Week 3: VizQL

Introduction

If you’ve read through all of the Log Analytics series, then you’re in pretty good shape when it comes to understanding the ‘voice’ of your Tableau Server. For our final week, we’re going to talk about the Tableau Server workhorse: Data Engine.  Yes, it might not be the most friendly of formats but with the right Log Analytics tool and strategy, you’ll finally be able to understand: how uploads progress, how long queries take to execute, how many ‘TEMP.#Tableau…’ tables there are and more! It’s really an interesting log set simply because it gives you a good idea of how well users are managing their extract’s design. You should really never see excessive query times as the Tableau Data Engine is pretty well put together. However, as all things go, sometimes stuff just doesn’t work.

 

ts-logs-de-1
The Data Engine EKG

If you environment is both extract heavy and view heavy, following this log (and the others mentioned above) will allow you to tune your environment with ease. You can, as we do, have over 3500 users with 100s of extracts on one machine.

Where’s the Log

From the Tableau Server admin guide:

The Tableau Server log directory is C:\ProgramData\Tableau\Tableau Server\data\tabsvc\logs if you installed Tableau Server on drive C, unless otherwise noted in the table below.

Data Engine, then, is in the folder dataengine.

What does it do for me?

Again, from the Tableau Server admin guide:

‘There will be a tdeserver log file for each day with information about data extracts and queries, and responses to VizQL server requests.’

But there’s so much more. Some things (but not all) we watch for with the Data Engine:

  • Upload size over time
  • Query Execution time > 10 seconds
  • ‘tdeserver: connection dead’ errors

For example, if you’re using a log analytics tool, you might be able to run a query like this:

ts-logs-de-4
graph of data engine activity

 

where(/UploadSetSize:  upload_guid=\d size=(?P<UploadSize>\d*)/) calculate(sum:UploadSize) timeslice(100)

ts-logs-de-5
text details of data engine activity

With the above query, we can get information the size of extracts being loaded. You can even trend this over to correlate with latency issues for users who may be experiencing ‘slowness’. And you do this by picking out the session IDs from the log and tying that back to the vizqlserver/logs directory of files (see how it all connects 🙂 ).

ts-logs-de-2
linking to another log via sessionid

Keep in mind, this is also how you’d want to leverage both alerting and tagging with your log analytics tool; if query time is consistently high and you’re seeing a lot of longer load times on dashboards, it may be time to investigate how users are designing their dashboards with their extracts.

ts-logs-de-0
query timeline

where(/time:(?P<Time>\d*)/) calculate(SUM:Time) timeslice(100)

As the above query will show, a consistent increase in execution time (or total time) may be something to investigate. Be aware, you can see the query being execution by searching for the ‘StatementPrepare’ portion of the log. Some of them can get pretty crazy.

ts-logs-de-3
Daily sessionid info, grouped by sess_guid

What should I be Alerted on?

Only those alerts which would require you to take action. Namely, these two:

  • ‘tdeserver: connection dead’ errors
  • Query Execution time > 60 seconds

So that’s it! Four weeks of Tableau Server Log Analytics and, believe it or not, there could be so much more! The four we’ve talking about cover a good portion of what you’d need to understand the basics of your environment.

And now that I’ve told you about consuming Tableau logs, why don’t you create your own?! Yes, in a series I’m currently working on, I’ll show you how to generate more detailed, user and security focused, logs related to Tableau Desktop (and Server).

 

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s