Title
Visualization of Predictive Modeling for Big Data Using Various Approaches When There Are Rare Events at Differing Levels
Document Type
Book Chapter
Identifier Data
https://doi.org/10.4018/978-1-5225-3142-5.ch015
Publisher
IGI Global
Publication Source
Handbook of Research on Big Data Storage and Visualization Techniques
Abstract
Many techniques exist for predictive modeling of a bivariate target variable in large data sets. When the target variable represents a rare event with an occurrence in the data set of approximately 10% or less, traditional modeling techniques may fail to identify the rare events. In this chapter, different methods, including oversampling of rare events, undersampling of common events and the Synthetic Minority Over-Sampling Technique are used to improve the prediction outcomes of rare events. The predictive models of decision trees, logistic regression and rule induction are applied with SAS Enterprise Miner (EM) to the revised data. Using a data set of home mortgage applications, misclassification percentages of a target variable with a rare event occurrence rate of 0.8% are obtained by running a multiple comparison node. The percentage is varied from 0.8% up to 50% and the results are compared to see which predictive method worked the best.
Comments
Is part of Handbook of Research on Big Data Storage and Visualization Techniques.