The Problem with LookML
Hi, my name is Reed Rawlings and I work as a solution engineer helping prospects evaluate Sigma. I am a former Looker analyst (Department of Customer Love) and Looker customer engineer.
If my time with new Looker customers could be summarized it would look like this, “Wow, LookML is pretty neat, but I’m worried about maintaining an entire code base just for analytics.” For customers who’d been using Looker for over a year it’d be, “How do I untangle all my LookML!”
Analytics is an ever expanding field and everyone, in every organization, needs something different. So, how can you contain the various and ever-growing needs of your business users in a single codebase? Spoiler alert, you can’t. At least, not without blowing up your LookML model.
Cut the Cruft
LookML or “Looker Modeling Language” is the proprietary code used throughout Looker for data modeling and analysis. Secure, governed, and robust in capability, LookML code is typically written and managed by a data engineer.
What Looker users would end up with is an ever growing repository of one-off metrics and dimensions, potentially only ever used a single time but because their complex structure could only be written in LookML.
Data engineers need to be responsive to exact these requests because they’re usually timely and highly important, however, they don’t want to give analysts or business users access to the codebase because any errors can be potentially catastrophic. So, they have to stop what they’re doing and put their focus on data requests as an intermediary for “self-service.”
Nonetheless, Looker had a solution. You could use Table Calculations, but those suffer some pretty serious limitations.
As you onboard more users, you end up with more data requests heading directly to LookML experts. Those requests add more LookML to your codebase, which we would affectionately call “LookML Bloat”—essentially more code for fewer answers.
The more bloat you have in Looker, the longer the LookML validator can take to run, which leads to another issue, validator pains.
Looker Validator Pains
LookML is codebased and every good codebase has a corresponding validator that lets you know whether you did something wrong. That’s incredibly helpful up until the moment it isn’t.
Enter this help article, “My LookML Validator is slow!” That’s right, as you add more to your codebase, your validator starts to slow down. “Slow down” can mean a lot of things, but in this case, it can include the time it takes to use the LookML you’ve written, crashing your browser, or even crashing your entire Looker instance.
The solution? A good old code refactoring. We just need to hope that the same person who wrote the original code is still in your organization and that all the ad-hoc requests aren’t running anything business critical. If not, you can always pay Looker to help you with this process.
This past summer, Looker released a new feature called, “New LookML Runtime.” It was supposed to speed up LookML validation, but for the first few days, the release prevented multiple instances from even working in LookML.
Model, Then Model Again
With the ever growing dominance of modeling tools like dbt, Looker users end up modeling their data twice. Once in their data warehouse and again in Looker.
The major issue with this is that dbt users only need to know SQL. Modeling in Looker requires you to know SQL & LookML. If you’re thinking about future-proofing your analytics, there is a much larger pool of SQL users than LookML users. If you already use dbt, portions of LookML become redundant. Yet, you still need to maintain both to make Looker work.
Aside from the issues above, data engineers’ time is quite valuable. Having them do redundant work is an artificial cost that no organization needs to maintain.
Enter Sigma’s approach
Sigma takes a no-code approach and can connect to data without an intermediary modeling layer. With Sigma, lookups and CSV uploads are approachable for business users. Sigma workbooks do not create inextricable cruft by being flexible without additional code creation. Sigma leverages transformation performed in the cloud data warehouse and any dependencies are easily traceable and well documented.
