Cape

Provenance and intervention-based techniques have been used to explain surprising high or low outcomes of aggregation queries based on the outcome’s provenance. However, such techniques may miss interesting explanations emerging from data that is not in the provenance. For instance, an unusually low number of publications of a prolific researcher in a certain venue in a year can be explained by an increase in his publication in another venue in the same year. In this project we investigate how to mine patterns that describe inherent trends in the data and to use these patterns to identify potential causes for an outcome of interest.

As an initial contribution we have developed a novel system called Cape (Counterbalancing with Aggregation Patterns for Explanations) for explaining outliers in aggregation queries through counterbalancing. That is, explanations are outliers in the opposite direction of the outlier of interest. Outliers are defined w.r.t. patterns that hold over the data in aggregate. We have developed efficient methods for mining such aggregate regression patterns (ARPs) and have demonstrated how to use ARPs to generate and rank explanations.

Collaborators

Sudeepa Roy - Duke University
Zhengjie Miao - Duke University

Publications

Going Beyond Provenance: Explaining Query Answers with Pattern-based Counterbalances
Zhengjie Miao, Qitian Zeng, Boris Glavic and Sudeepa Roy
Proceedings of the 44th International Conference on Management of Data (2019), pp. 485–502.
- doi
- pdf
- slides
- video
- details
```
@inproceedings{MZ19,
  author = {Miao, Zhengjie and Zeng, Qitian and Glavic, Boris and Roy, Sudeepa},
  booktitle = {Proceedings of the 44th International Conference on Management of Data},
  keywords = {Provenance; Explanations; Cape},
  pages = {485--502},
  pdfurl = {http://cs.iit.edu/%7edbgroup/assets/pdfpubls/MZ19.pdf},
  doi = {10.1145/3299869.3300066},
  video = {https://av.tib.eu/media/42903},
  projects = {Explanations beyond Provenance},
  slideurl = {https://www.slideshare.net/lordPretzel/2019-sigmod-going-beyond-provenance-explaining-query-answers-with-patternbased-counterbalances},
  title = {Going Beyond Provenance: Explaining Query Answers with Pattern-based Counterbalances},
  venueshort = {SIGMOD},
  year = {2019}
}
```
Provenance and intervention-based techniques have been used to explain surprisingly high or low outcomes of aggregation queries. However, such techniques may miss interesting explanations emerging from data that is not in the provenance. For instance, an unusually low number of publications of a prolific researcher in a certain venue and year can be explained by an increased number of publications in another venue in the same year. We present a novel approach for explaining outliers in aggregation queries through counterbalancing. That is, explanations are outliers in the opposite direction of the outlier of interest. Outliers are defined w.r.t. patterns that hold over the data in aggregate. We present efficient methods for mining such aggregate regression patterns (ARPs), discuss how to use ARPs to generate and rank explanations, and experimentally demonstrate the efficiency and effectiveness of our approach.
details
CAPE: Explaining Outliers by Counterbalancing
Zhengjie Miao, Qitian Zeng, Chenjie Li, Boris Glavic, Oliver Kennedy and Sudeepa Roy
Proceedings of the VLDB Endowment (Demonstration Track). 12, 12 (2019) , 1806–1809.
- doi
- pdf
- details
```
@article{MZ19a,
  author = {Miao, Zhengjie and Zeng, Qitian and Li, Chenjie and Glavic, Boris and Kennedy, Oliver and Roy, Sudeepa},
  date-modified = {2019-08-02 09:14:13 -0500},
  journal = {Proceedings of the VLDB Endowment (Demonstration Track)},
  keywords = {Outliers; Intervention; Cape; Explanations},
  pdfurl = {http://www.vldb.org/pvldb/vol12/p1806-miao.pdf},
  projects = {Explanations beyond Provenance},
  pages = {1806-1809},
  volume = {12},
  issue = {12},
  doi = {10.14778/3352063.3352071},
  title = {CAPE: Explaining Outliers by Counterbalancing},
  venueshort = {{PVLDB}},
  year = {2019}
}
```
In this demonstration we showcase Cape, a system that ex- plains surprising aggregation outcomes. In contrast to previous work which relies exclusively on provenance, Cape applies a novel approach for explaining outliers in aggregation queries through counterbalancing (outliers in the opposite direction). The foundation of our approach are aggregate regression patterns (ARPs) based on which we defined outliers, and an efficient explanation generation algorithm that utilizes these patterns. In the demonstration, the audience can run aggregation queries over real world datasets, and browse the patterns and explanations returned by Cape for outliers in the result of such queries.
details