Why use both LookML and ELT tools in your data analytics stack

Background

LookML is a robust device that applies enterprise logic and governance to enterprise information analytics. Nonetheless, LookML’s capabilities are sometimes conflated with these of in-warehouse “ELT” transformation instruments like Dataform and DBT. 

As these instruments seem like related in nature, it’s typically thought that customers want to decide on one over the opposite. This submit outlines why clients ought to be utilizing each LookML and ELT instruments of their information analytics stack, with a particular deal with the significance of LookML. In a  follow-up article, we’ll cowl how you must architect what you are promoting logic and transformations between the LookML and ELT layers.

Fast background on LookML

In case you are new to LookML, it would be best to take a look at this video and assist heart article to get extra acquainted. However to shortly summarize:

LookML is a code-based modeling layer primarily based on the ideas of SQL that:

• Permits builders to obfuscate the complexity behind their information and create an explorable atmosphere for less-technical information personas

• Brings governance and consistency to analytics as a result of all queries are primarily based off of the LookML mannequin, appearing as a “single supply of reality”

• Permits trendy growth by means of its git-integrated model management

LookML can energy third-party instruments

LookML was initially designed to allow analytics within Looker’s UI (or in customized information purposes utilizing Looker’s APIs). Looker has continued to evolve and introduced a number of LookML integrations into different analytics instruments, enabling clients so as to add governance and belief to extra of their analytics, no matter which consumer interface they’re utilizing. 

Elevated adoption of “ELT” Transformation instruments

Over the previous few years, many organizations have adopted in-warehouse ELT transformation instruments, like Dataform (from Google) and DBT, as an alternative choice to conventional ETL processes. ETL sometimes transforms information earlier than it’s loaded into an information warehouse. ELT instruments take a extra simplified and agile method by remodeling information after it’s loaded within the warehouse. In addition they adhere to trendy growth practices.

Similarities with LookML

On the floor, traits of those ELT instruments sound similar to these of LookML. Each:

• Are constructed on the foundations of SQL

• Allow reusable and extendable code

• Assist outline standardized information definitions

• Are Git version-controlled and collaborative

• Have devoted IDEs with code validation and testing

The deeper worth of LookML

LookML provides three important capabilities to your group that can’t be accomplished solely with an ELT device:

1. Versatile metrics and exploratory evaluation

2. Consistency and governance within the “final mile” of your analytics

3. Agile growth and upkeep of your BI layer

Versatile metrics and exploratory evaluation

Many groups try to outline and govern their information solely utilizing their most acquainted ELT device. One purpose you must keep away from that is associated to aggregated metrics (also called “details” or “measures”), particularly non-additive and semi-additive metrics. ELT instruments should not designed to effectively help all these metrics, with out constructing numerous pointless tables.

Sorts of BI metrics

Versatile Metrics
A versatile metric is a metric that’s dynamically calculated primarily based on the scope of the request from the consumer. For instance, let’s say what you are promoting needs to report on the Every day Lively Customers in your web site, a non-additive metric. In the event you’re working inside your ELT device, you could say: “Hey, that is straightforward! I can construct a brand new desk or view.”