Introduction to Causal Impact

Share this post

Understand what is CausalImpact and how it can be used to understand the causal effect of an event in the absence of an experiment.

What is CausalImpact?

CausalImpact is a package created by Kay H. Brodersen that uses Bayesian statistics to infer the causal effect of an event.

It is commonly used in pre-post experiments or to evaluate the effect of a past event.


Subscribe to my Newsletter


How CausalImpact Works?

CausalImpact (CI) uses predictive analytics to estimate what would have happened to a variable in the absence of an event.

Then, it compares the actual results to the estimation.

In other words, it tries to estimate the counterfactual and compare it to the observations.

CausalImpact can be used in two ways

  1. Simple pre-post experiment
  2. Compared against control groups

Simple pre-post experiment

CausalImpact can be used with a single dataset (y) without any control group.

In which case, using seasonality and other factors, Causalimpact will make a prediction based on previously observed data in the absence of an event.

Then, it will compare the actual observation to the predicted data.

This image has an empty alt attribute; its file name is image-57.png
Source: Bigdata spain

Compared against control groups

CausalImpact can be used with a test dataset (y) with 1 or multiple control groups (X1, X2, …, Xn).

In this case, it will use the control groups to try to make a better estimation of the data in the absence of the event.

Source: Bigdata spain

How to use CausalImpact

Below are two tutorials to help use CausalImpact in R (original package) or Python (wrapper).


Intepret the CausalImpact Graph

Regardless of the style applied to your graphs, it should look something like this.

You Might Also Like  Python CausalImpact (pyCausalImpact) and GSC - SEO Experiment

In the graph above:

  • the vertical dotted line represents the intervention date
  • The dotted line represents the boundary where above means positive variations and below negative variations.
  • The shaded area represents the confidence interval.

The top graph shows the observed data against the predicted values.

The middle graph shows the point effect of one against each other

The bottom graph shows the cumulative effect of the confidence interval.

Statistical Significance

Given a 95% confidence interval, Causal Impact will tell you whether your experiment is statistically significant.

In the graph, statistical significance is reached whenever the shade area goes above or below the 0 line (like in the example below).

Science Behind CausalImpact

Citations

CausalImpact Package Documentation

CausalImpact Essay

Enjoyed This Post?