Use cases of DPL

Parse.ly’s Data Pipeline is useful to a number of roles, including CTOs, CIOs, Data Scientists, Data Engineers, BI Analysts, SQL Analysts, and anyone else who derives value from a unified real-time stream of user, web, and mobile engagement data.

For business intelligence teams

BI teams need to understand their users and business at a fine-grained level. This can be helpful for corporate/executive reporting functions, or to guide other departments like product, audience development, content, or marketing.

Only raw data can enable ad-hoc reporting, and Parse.ly’s Data Pipeline provides reliable and foundational infrastructure for this purpose.

For example, customers often integrate Parse.ly’s Data Pipeline with their own ETL before loading the data into Amazon Redshift or BigQuery. They then use a tool like Looker to provide data exploration and dashboard interfaces for their teams.

Below is an example dashboard built using Parse.ly’s Looker Data App, which is itself based on Parse.ly’s standard event schema. The Looker Data App (built with LookML) can recreate many of the queries that power Parse.ly’s real-time audience dashboards, but within Looker itself, with full control and customizability.

For SQL experts

The world’s most popular relational database engines, like MySQL and Postgres, used to be a poor fit for raw event analytics data, simply due to scale. But in the last few years, a number of cloud SQL offerings have emerged that make analyzing terabytes of raw event data not only possible, but easy.

Amazon Redshift and Google BigQuery are two market leaders, both fully compatible with Parse.ly’s Data Pipeline. Integration with these systems is often no more than a few lines of code. This is because Parse.ly’s data formats are specifically optimized for clean integration with their bulk loading and stream loading tools.

Organizations with SQL expertise can use a tool like Periscope to query raw data. Dashboard components are built up from a raw SQL query run against Parse.ly events.

Here is an example Periscope dashboard built from Parse.ly’s raw data schema.

For data scientists

Data Science is the combination of data analysis, statistics, and programming. Parse.ly’s Data Pipeline supports interactive data exploration environments such as:

Jupyter, for Python users
R Studio, for R users

There are also hosted environments that work well, such as:

Databricks Community Edition, for Scala/Python/R users
Mode Analytics, for Python/SQL users

Here is an example Mode Analytics SQL sheet running a query against Parse.ly raw data that has been synced up to BigQuery.

For data engineers

Parse.ly data can provide a great starting point for leveraging open source “big data” technologies, such as:

Map/Reduce: Hadoop, Pig, HDFS
In-Memory: Spark, SparkSQL
Streaming: Storm, Kafka
Log Analytics: Hive, HBase, Cassandra
Document Stores: MongoDB, Elasticsearch
MPP Databases: PrestoDB, Drill, Druid, Impala

Data Pipeline is also a great fit for public clouds and their associated analytics technologies, such as:

Amazon’s: EMR and Redshift
Google’s: Dataproc and BigQuery
Microsoft’s: Azure Spark and HDInsight

For product teams

Even without a formal analytics, data science, or business intelligence practice, raw analytics data can be one of the best ways to evolve a product. A reliable Data Pipeline can also create virtuous product feedback loops, such as:

personalization
alerting
internal usage dashboards
loyal user targeting
email and notifications

For business intelligence teams

For SQL experts

For data scientists

For data engineers

For product teams

Next steps