Visualization of Predictive Modeling for Big Data Using Various Approaches When There Are Rare Events at Differing Levels

Document Type

Book Chapter


Is part of Handbook of Research on Big Data Storage and Visualization Techniques.

Identifier Data



IGI Global

Publication Source

Handbook of Research on Big Data Storage and Visualization Techniques


Many techniques exist for predictive modeling of a bivariate target variable in large data sets. When the target variable represents a rare event with an occurrence in the data set of approximately 10% or less, traditional modeling techniques may fail to identify the rare events. In this chapter, different methods, including oversampling of rare events, undersampling of common events and the Synthetic Minority Over-Sampling Technique are used to improve the prediction outcomes of rare events. The predictive models of decision trees, logistic regression and rule induction are applied with SAS Enterprise Miner (EM) to the revised data. Using a data set of home mortgage applications, misclassification percentages of a target variable with a rare event occurrence rate of 0.8% are obtained by running a multiple comparison node. The percentage is varied from 0.8% up to 50% and the results are compared to see which predictive method worked the best.

This document is currently not available here.