Cracking the JSON Code

A Guide to Managing and Analyzing JSON
with Snowflake and Sigma

A New Era for Data

The era of Big Data—and big data analytics—is upon us. By 2025 the global datasphere will grow to an estimated 175 zettabytes. For context, from the dawn of the internet to 2016, the web created a single zettabyte of data.

Image Source: Forbes

Of course, internet traffic is only a single slice of the data pie created and stored around the world — which includes all personal and business data. Today, the world sits somewhere between 40-50 zettabytes of total data. Which begs the question: what do we do with all this data? And what good will come from the constant collection of data across the web, personal devices, Internet of Things (IoT), and more? If you said, “analyze it for insights,” you’re on the right track. Unfortunately, up to 73% of all enterprise data goes unused. As more organizations aim to become data-driven, it’s imperative to make better use of the data they collect.

A good place to start is JSON. Since it arrived on the scene in 2001, JSON has become the preferred data interchange format for mobile devices, web applications, online services, and sensors. This includes some of today’s most popular websites like Facebook and Google — and the fast-growing market of wearables and IoT devices. These services and devices produce an unprecedented amount of data in our digital economy. Unstructured and semi-structured data (like JSON) now make up 80% of the data collected by enterprises. And that number is only expected to grow in the coming years.

All this data is a potential treasure trove for companies that can harness it effectively. But combing through JSON in real time to find patterns, emerging trends, and insights has historically taken significant time and resources.

JSON analysis presents challenges for companies that want to make sense of this vast repository of information. JSON can’t be stored, managed, and analyzed as quickly as structured data formats — which has data experts leaving valuable insights on the table. And if you’re a marketer, salesperson, or product leader, you can’t extract and analyze JSON without significant hand-holding from your colleagues on the data team. Even for those with a background in data science, the process of parsing JSON is cumbersome.

With Snowflake+Sigma, it’s possible for JSON analysis to extend beyond data and IT teams. Domain experts can now parse JSON, join it with other structured data, and generate insights to drive better decisions and get a leg up on the competition.

Enjoying this eBook?

Save it for later.

Download PDF

The Rise of JSON

For those less familiar with JSON, or JavaScript Object Notation, it is a schema-less, text-based representation of structured data that is based on key-value pairs and ordered lists. JSON has become extremely popular over the last decade. Today, any time web applications communicate, it’s usually through JSON. Internet giants like Google, Facebook, and Twitter all rely on JSON to expose data through APIs.

While there are other semi-structured data formats (such as XML and AVRO), none are more popular than JSON. It’s become the de-facto way that programs interchange data in the modern era. So how – and why – did this happen?

The first JSON message was sent in 2001 from a server to a Douglas Crockford’s laptop in Chip Morningstar’s garage, which was the birthplace of their small startup called State Software.

2001

The first JSON message was sent from a server to Douglas Crockford’s laptop in Chip Morningstar’s garage.


<html><head><script>
    document.domain = 'fudco';
    parent.session.receive(
        { to: "session", do: "test",
          text: "Hello world" }
    )
</script></head></html> 
The first JSON message sent in 2001

Crockford and Morningstar realized they could sidestep an HTML frame and send themselves form fill data. They found that this new method of data interchange was an efficient way to communicate data between servers, and even build a database. It was simple and represented an intersection of all modern programming languages.

They decided to call this new interchange format JSON, and registered json.org to get the word out. Soon other programmers started implementing JSON code across the web in various applications and sharing their work. Here’s a video where Crockford recounts the early days of JSON.

Want to keep reading?

Learn how Sigma and Snowflake can help you crack the JSON code.