A Representative Sampling Method for the Peer Encouragement Design of Network Experiments

Marketers are increasingly targeting the influential individuals on online social networks to leverage the social influence exerted by these individuals on their connections. To causally identify the direct and indirect effects of such a marketing intervention, researchers often use randomized experiments with the “peer encouragement design” in which a sample of focal individuals (i.e., egos) are randomly split into the treatment group (with the marketing intervention) and the control group. Then, the outcomes of the two groups of egos are compared to identify the direct effect, and the outcomes of the egos’ connections (i.e., alters) are compared to identify the indirect effect of the marketing intervention. One challenge with this design is the treatment contamination or leakage. A commonly adopted approach to mitigate this concern is to exclude the contaminated egos/alters in the inference of the treatment effects. In this research, we demonstrate that simply excluding the contaminated nodes can lead to two potential problems. First, the reduced samples may not represent the population because individuals with more connections are more likely to be contaminated and hence excluded, which reduces the heterogeneity in the sample and leads to biased estimates of direct and indirect treatment effects. Second, the resulting sample size is limited and unpredictable, which lowers the test power and efficiency of an experiment. These issues are particularly concerning in situations when population network is dense, or when the treatment is costly. To address these issues, we propose a representative sampling method to generate ego network samples through the Bayesian Metropolis algorithm.. Through simulations, we show that our proposed method enables more precise and reliable estimation/inference of the average treatment effects compared to the commonly used “sampling-excluding” approach. More importantly, our method can precisely capture the sources and amounts of the heterogeneous treatment effect and allow for accurate out-of-sample prediction of treatment responses on unseen individuals in the population. The proposed representative sampling method is easy to implement in practice and can be especially beneficial to companies aiming at targeting influentials on online social networks.