In-Memory or Live Data: Which Is Better?

Discussion in 'Tableau' started by tableau, Oct 24, 2015.

  1. tableau

    tableau Well-Known Member

    The short answer is both.

    Companies today are using both to deal with ever-larger volumes of data. For the business user, analyzing large data can be challenging simply because traditional tools are slow and cumbersome. Some companies are turning to in-memory BI solutions, which let users extract set of data and take advantage of the computing power on their local machine to speed up reporting and take query load off their transactional systems. Others are investing in fast, analytical databases that are distinct from their transactional systems.

    There are times when you need to work within the comfort of your own PC without touching the database. On the other hand, sometimes a live connection to a database is exactly what you need. The most important thing is not whether you choose in-memory or live, but that you have the option to choose.

    Let’s look at some scenarios where in-memory or live data might be preferable.

    In-Memory Data is Better…
    When your database is too slow for interactive analytics

    Not all databases are as fast as we’d like them to be. If you’re working with live data and it’s simply too slow for you to do interactive, speed-of-thought analysis, then you may want to bring your data in memory on your local machine. The advantage of working interactively with your data is that you can follow your train of thought and explore new ideas without being constantly slowed down waiting for queries.

    When you need to take load off a transactional database

    Your database may be fast, it may be slow. But if it’s the primary workhorse for your transactional systems, you may want to keep all non-transactional load off it. That includes analytics. Analytical queries can tax a transactional database and slow it down. So bring a set of that data in-memory to do fast analytics without compromising the speed of critical business systems.

    When you need to be offline

    Until the Internet comes to every part of the earth and sky, you’re occasionally going to be offline. Get your work done even while not connected by bringing data in-memory so you can work with it right on your local machine. Just don’t forget your power cord or battery— you’ll still need that!

    A Live, Direct Data Connection is Better…
    When you have a fast database

    You’ve invested in fast data infrastructure. Why should you have to move that data into another system to analyze it?

    You shouldn’t. Leverage your database by connecting directly to it. Avoid data silos and ensure a single point of truth by pointing your analyses at a single, optimized database. You can give business people the power to ask and answer questions of massive data sets just by pointing to the source data. It’s a simple, elegant and highly secure approach.

    When you need up-to-the minute data

    If things are changing so fast that you need to see them in real-time, you need a live connection to your data. All your operational dashboards can be hooked up directly to live data so you know when your plant is facing overutilization or when you’re experiencing peak demand.

    And the Best Choice? Both, of Course.
    Even better is when you don’t have to choose between in-memory and live connect. Instead of looking for a solution that supports one or the other, look for one that supports choice. You should be able to switch back and forth between in-memory and live connection as needed. Scenarios where this is useful:

    • You want to use a sample of a massive data set to find trends and build your analysis. You bring a 5% sample of the data in-memory, explore it and create a set of views you want to share. Then you switch to a live connection so your reports are working directly against all the data. Publish your views, and now your colleagues can interact with your analysis and drill down to the part of the data most relevant to their work.
    • You’re flying to New York and want to do some analysis on the plane. You bring your entire data set, several million rows, into your local PC memory and work with it offline. When you get to New York, you reconnect to the live data again. You’ve done your analysis offline and in-memory, but you are able to switch back to a live connection with a few clicks.
    Neither in-memory nor live connect is always the right answer. If you’re forced to choose, you’ll lose something every time. So don’t choose—or rather, choose as you go. Bring your data in-memory, then connect live. Or bring recent data in-memory and work offline. Work the way that makes sense for you.
  2. Loading...

Share This Page