Skip to content

Commit

Permalink
Rename Snowplow Insights to Snowplow BDP (close #115)
Browse files Browse the repository at this point in the history
  • Loading branch information
Paul Boocock authored and paulboocock committed Nov 12, 2021
1 parent 9bf46f3 commit ba0d08c
Show file tree
Hide file tree
Showing 4 changed files with 11 additions and 11 deletions.
2 changes: 1 addition & 1 deletion .scripts/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,7 @@ It's best to avoid committing these to source control - however doing so is less

Runs a config json file (examples found in the `configs` folder of each model) - which specifies a list of playbooks to run.

Note that this script does not enforce dependencies, rather runs the playbooks in order of appearance. Snowplow Insights customers can take advantage of dependency resolution when running jobs on our Orchestration services.
Note that this script does not enforce dependencies, rather runs the playbooks in order of appearance. Snowplow BDP customers can take advantage of dependency resolution when running jobs on our Orchestration services.

**Arguments:**

Expand Down
16 changes: 8 additions & 8 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,21 +2,21 @@

![snowplow-logo](media/snowplow_logo.png)

Snowplow is a scalable open-source platform for rich, high quality, low-latency data collection. It is designed to collect high quality, complete behavioural data for enterprise business.
Snowplow is a scalable open-source platform for rich, high quality, low-latency data collection. It is designed to collect high quality, complete behavioral data for enterprise business.

# Snowplow Pipeline Overview

![snowplow-pipeline](media/snowplow_architecture.png)

The [Snowplow trackers][tracker-docs] enable highly customisable collection of raw, unopinionated event data. The pipeline validates these events against a JSONSchema - to guarantee a high quality dataset - and adds information via both standard and custom enrichments.
The [Snowplow trackers][tracker-docs] enable highly customizable collection of raw, unopinionated event data. The pipeline validates these events against a JSONSchema - to guarantee a high quality dataset - and adds information via both standard and custom enrichments.

This data is then made available in-stream for real-time processing, and can also be loaded to blob storage and data warehouse for analysis.

The Snowplow atomic data acts as an immutable log of all the actions that occurred across your digital products. The data model takes that data and transforms it into a set of derived tables optimized for analysis. [Visit our documentation site][docs-what-is-dm] for further explanation on the data modeling process.

# Try Snowplow

This repo contains data models which are relevant to users who already have a full Snowplow pipeline running (which can be done Open Source or via our [Snowplow Insights](https://snowplowanalytics.com/snowplow-insights/) service).
This repo contains data models which are relevant to users who already have a full Snowplow pipeline running (which can be done Open Source or via [Snowplow BDP](https://snowplowanalytics.com/snowplow-bdp/)).

If you don't have a pipeline yet, you might be interested in finding out what Snowplow can do by setting up [Try Snowplow](https://try.snowplowanalytics.com/?utm_source=github&utm_medium=post&utm_campaign=try-snowplow).

Expand All @@ -35,7 +35,7 @@ Documentation for the data models can be found on [our documentation site][docs-

# Prerequisites

These models are written in a format that is runnable via [SQL-runner][sql-runner] - available for download as a zip file from [Github Releases][sql-runner-github]. The bigquery model requires >= v0.9.2, and the Snowflake model requires >= v0.9.3 of sql-runner.
These models are written in a format that is runnable via [SQL-runner][sql-runner] - available for download as a zip file from [Github Releases][sql-runner-github]. The BigQuery model requires >= v0.9.2, and the Snowflake model requires >= v0.9.3 of sql-runner.

Those who don't wish to use sql-runner to run models can use the -t and -o flags of the run_config.sh script to output the pure sql for a model according to how it has been configured for sql-runner.

Expand All @@ -48,13 +48,13 @@ cd .tests
pip3 install -r requirements.txt
```

# Quickstart
# Quick start

To run a model and tests end to end, run the `.scripts/e2e.sh` bash script.

![end-to-end](media/e2e.gif)

For a quickstart guide to each individual model, and specific details on each module, see the README in the model's database-specific folder (eg. `web/v1/redshift`).
For a quick start guide to each individual model, and specific details on each module, see the README in the model's database-specific folder (eg. `web/v1/redshift`).

For detail on the structure of a model, see the README in the model's main folder (eg. `web/v1`).

Expand All @@ -64,9 +64,9 @@ For detail on using the helper scripts, see the README in `.scripts/`

## Using SQL-runner

### Snowplow insights
### Snowplow BDP

Snowplow Insights customers can configure jobs for SQL-runner in production via configuration files. [See our docs site for details on doing so](https://docs.snowplowanalytics.com/docs/modeling-your-data/configuring-and-running-data-models-via-snowplow-insights/). The `configs/datamodeling.json` file in each model is an example configuration for the standard model. The `configs/example_with_custom.json` file is an example configuration with a customisation.
Snowplow BDP customers can configure jobs for SQL-runner in production via configuration files. [See our docs site for details on doing so](https://docs.snowplowanalytics.com/docs/modeling-your-data/configuring-and-running-data-models-via-snowplow-bdp/). The `configs/datamodeling.json` file in each model is an example configuration for the standard model. The `configs/example_with_custom.json` file is an example configuration with a customization.

### Open Source

Expand Down
2 changes: 1 addition & 1 deletion web/v1/bigquery/sql-runner/sql/custom/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ Each standard module also contains a `99-{module}-complete` playbook, which comp

The simplest means of assuring that everything is run in the correct order is to run all relevant `99-complete` steps at the end of the model, in the order demonstrated in the `../../configs/example_with_custom.json` configuration.

Snowplow Insights customers running on orchestration need only ensure that dependencies are specified in the config file.
Snowplow BDP customers running on orchestration need only ensure that dependencies are specified in the config file.

## Producing custom tables

Expand Down
2 changes: 1 addition & 1 deletion web/v1/snowflake/sql-runner/sql/custom/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@ Each standard module also contains a `99-{module}-complete` playbook, which comp

The simplest means of assuring that everything is run in the correct order is to run all relevant `99-complete` steps at the end of the model, in the order demonstrated in the `../../configs/example_with_custom.json` [configuration](../../configs/example_with_custom.json).

Snowplow Insights customers running on orchestration need only ensure that dependencies are specified in the config file.
Snowplow BDP customers running on orchestration need only ensure that dependencies are specified in the config file.

## Producing custom tables

Expand Down

0 comments on commit ba0d08c

Please sign in to comment.