代写Bank Customer Churn Analysis Based on Decision Trees代做留学生SQL语言程序
- 首页 >> Database作业Masters Thesis
Bank Customer Churn Analysis Based on Decision Trees
Course code and name: |
F21RP-Research Methods and Project Planning |
Type of assessment: |
Individual |
Coursework Title: |
Bank Customer Churn Analysis Based on Decision Trees |
Abstract
Rapid development of a new information network changes the quality of the traditional business model. Especially in the financial arena, customers have higher expectations for banks to meet the needs of products and services, and at the same time, the competition will intensify. In such a setting, customer loyalty has turned into an important indicator of the ability of building and retaining the customer relationship that enables banks to further their competitive advantage.
On this point, the research is based on basic data of the customers of the bank and is based on the needs of business marketing scenarios, abstracting the problem model for the solution of individual customer churn. It is processed through the use of feature engineering combined with the customer labeling system applicable data characteristics of the bank's customer churn early warning model. The churn model that is applicable to the effective-value customers of the bank is being built using the decision tree algorithm, through which one can identify the areas of bank products and strategies in marketing, maintenance of customers, and service. Furthermore, it develops a model that can be applied in banks to come up with an effective value customer churn model, whereby it suggests the areas related to bank products and marketing strategies, customer maintenance, and services.
Keywords:Decision Tree, Bank Customer Churn, Unbalanced Data,Hybrid Feature Selection.
Customers are the root of the bank, is its survival and development of the fundamental resources. Presently, as the time has passed, the competition between banks for customer resources has gotten tighter, and attention in the banking industry started to shift towards customer needs, where efforts focused on satisfying customer needs for improved customer service. In this context, the maintenance of bank customers and improving customer loyalty and dependence is very critical. With the gradual development of modern technology, there has been a fundamental change in the operation of the modern financial industry. The customer relationship management system is an important tool that not only serves for batch maintenance and operation of customers but also offers great possibilities for the in-depth analysis of data,
mining of potential demand, precision marketing, and control of marketing processes.
1.2 Shortcomings of Existing Studies
Further on the statistical analysis, it shows that the individual customer becomes the principal customer group for banks. This group is maintained in its functioning and its expansion, so it has become the one with the most priority in the daily operation of commercial banks. With the upgrading of customers' consumption demand, space is given to customers in the degree of requirement for financial services, the difference of supply and demand patterns in the financial market, and the customers of the bank customer base change their kind of increase due to a combination of factors. On the other hand, customer attrition would not only escalate the bank marketing expenses and opportunity costs but also give a bad reputation to the bank [Wang, W.Q., Yao, R., Liu, C., et al., 2014]. However, studies have shown that while for banking, the line customer churn has a huge impact on profits, reduction of customer churn by 5% can bring 30% to 85% increase in profits. The cost of developing a new customer is five to seven times more than the cost of retaining one, and the success rate of developing customers is only one-sixteenth of keeping customers [Xiao, J., Liu, D.H., and He, C.Z., 2012].
Research the customer churn early warning model, find out the influencing key factors of customer churn, and then apply them to actual engineering based on retail customer labeling data. Use the method of classification prediction to find the customer churn prediction, or the key factors that influence customer churn. Then, establish a model to effectively forecast the possibility of customer churn and formulate the corresponding recovery measures so that the customers will not be lost. This is how banks can enhance their core competitiveness.
1.4 Project Research Objectives
The main objective of the research would be to cover, in detail, the customer churn behavior of the bank by analyzing loads of data and then predicting, using the latest modeling techniques, and finally making an effective customer retention strategy to support continued business development for the bank, along with growth in profits. The specific objectives of the study can be broken down as follows:
O1. Identify key predictors of customer churn
By collecting, analyzing, and processing the customer data in the bank, including but not limited to the demographic characteristics, types of accounts, transaction behaviors, credit histories, and service interaction records of the customer. This will use descriptive statistics and exploratory data analysis while seeking to find probable major influencing variables of customer decision churn.
O2. Model Evaluation and Performance Optimization
The performance metrics can be accuracy, recall, F1 score, area under ROC, among others. Further, we will study how model performance can be optimized using techniques like Pruning, Integrated Learning methods (Random Forest and Boosted Tree).
O3. Construct and optimize a decision tree predictive model
Now, you are to deploy a decision tree algorithm for the building of your customer churn predictive model. This model is going to allow you to really say that so many customers are potential churn customers, and it really drives key factors' predictive power for churn. Model building involves selection of the right parameters, cross-validation to avoid overfitting, and checking the model's accuracy and robustness with the help of training and test sets.
O4. Develop data-driven customer retention based strategies
Propose strategies based on the model insights that could reduce customer churn. This may include enhancing the experience of customer service, changing pricing strategies, targeting delivery of customized marketing campaigns, and optimization of the product portfolio. This will help in the recommendation of the appropriate strategy advice that ensures the needs and preferences of the different segments are maximized.
With these nuanced research objectives, the present study tries to enhance not only customer retention for banks but tries to provide a methodological reference of applying the decision tree model to a complex data environment. The same can assist banks in better understanding of the service optimization needs in the customer, and simultaneously compete well in the highly competing financial market.
1.5 Possible innovations and shortcomings
1.5.1 Possible innovations
1. In fact, the feature selection strategy of the dataset in the research project is the strong mining capacity with regard to hybrid. The experiment proves that the method mines feature information and intrinsic connection, which is very effective to help enhance the effect of model prediction.
2. Research finds that the two-stage serial combination applies the two-stage serial combination model to achieve the two-stage combination model. The results from the experiment applied show the combination model to be effectively able to enhance the predictive ability of the model.
1.5.2 Possible shortcomings
1. The selected feature processing mainly relies on the original data, and the selection chooses without taking into consideration the actual significance of the variables. They will probably miss some of the most important features.
2. The presented research project adopts a two-stage serial combination model in the construction of a combination model and makes its comparison with the single model. Later, other forms of the combination are tried for the comparative analysis.
2.1 Review of bank customer churn prediction methods
The study of customer churn prediction methods encompasses various research efforts that provide insights into how organizations can mitigate customer turnover. Colgate et al. explored the churn behaviors of tertiary students within Irish financial services using questionnaires to analyze churn causes and patterns. They also examined how these factors align with financial policies and marketing strategies, emphasizing the need to tailor these aspects to reduce churn [Colgate, M., Stewart, K., Kinsella, R., 1996].Walsh et al. utilized structural equation modeling based on surveys from 462 customers of a German utility company. Their findings suggest that enhancing customer satisfaction through targeted marketing can significantly reduce churn. They also highlighted how corporate reputation and customer satisfaction impact churn, offering actionable insights for refining marketing strategies [Walsh, G., Dinnie, K., Wiedmann, K., 2006].Sohn et al. developed a competitive risk model that incorporates customer characteristics, which was particularly aimed at the mobile telecommunications sector in South Korea. Their study was catalyzed by the introduction of mobile number portability, and they proposed management guidelines based on their findings to help companies better handle customer retention under this new regime [Sohn, S.Y., Lee, J.K., 2008].Chen et al. crafted a customer value model using logistics industry data, identifying key factors that lead to the loss of valuable customers.
Their research provides strategic recommendations for customer management, aiming to enhance retention and prevent churn [Chen, K., Hu, Y.H., Hsieh, Y.C., 2015].These studies collectively advance the understanding of customer churn and offer a solid foundation for companies to develop targeted strategies to enhance customer loyalty and retention [Chen, K., Hu, Y.H., Hsieh, Y.C., 2015].Hwang H et al. posited that customer churn could be assessed from the perspective of the customer's value to the bank and their potential future revenue generation. This approach focuses on the economic contributions of customers to predict churn, advocating a value-based strategy to identify high-risk customers [Hwang, H., Jung, T., Suh, E., 2004]. Lu N et al. implemented logistic regression as a base learner in their churn prediction models, enhancing model accuracy through the use of boosting techniques. Their research targets the creation of early warning systems tailored to different bank customer groups, demonstrating the effectiveness of adaptive learning methods in improving predictive accuracy [Lu, N., Lin, H., Lu, J., et al., 2014]. Vafeiadis et al. examined several common machine learning algorithms for classifying potential churners but found no definitive best learner due to the complex interplay of factors like data type and distribution. Their work underscores the challenges in selecting the optimal machine learning approach in environments with diverse data characteristics [Vafeiad is, T., Diamantaras, K.I., Sarigiannid is, G., et al., 2015].
The existing literature primarily explores the use of integrated algorithms for predicting bank customer churn, with several studies incorporating advanced feature processing techniques like sampling and feature derivation. These techniques have proven effective at extracting valuable insights from the data, though the overall enhancement in model performance remains moderate. The studies suggest that while current methods are capable of identifying key indicators of churn, there remains a significant opportunity to explore combinatorial modeling approaches, which have been less utilized in this domain. This gap in research presents a potential area for further exploration to develop more robust and comprehensive predictive models.
2.2 Overview of modeling methods
Mozer et al. designed an early warning model for subscriber churn based on extensive US domestic subscriber data, totaling nearly 47,000 entries. This data encompassed a wide range of variables including consumption history, billing information, credit card data, application usage, and customer complaints. The researchers applied a variety of predictive modeling techniques, such as logistic regression, decision trees, neural networks, and boosting algorithms. The insights gained from the model were used to tailor subscriber incentives, aiming to improve retention rates and maximize operator profits. This model's effectiveness was not only theoretically proven but also practically verified in real business environments, demonstrating its applicability and impact on business operations [Mozer, M.C., Wolniewicz, R., Grimes, D.B., et al., 2000].Mohammed et al. focused on comparing decision tree and logistic regression models to determine their efficacy in predicting customer churn. Their study was grounded in empirical data from a mobile operator's business records. The findings indicated that decision trees offered superior performance over logistic regression models in this context, suggesting that decision trees might be more adept at handling the complexities and nuances involved in churn prediction in the telecom sector [Mohammed, H., Ali, T., Tariq, E., et al., 2015].Vafeiadis et al. utilized a public telecom customer dataset to perform a comparative analysis of five different algorithms using the Monte Carlo method, enhanced with boosting techniques. The algorithms tested included artificial neural networks, support vector machines, decision trees, plain Bayes, and logistic regression. Their comprehensive evaluation concluded that support vector machines provided the best results among the tested models, particularly in managing and interpreting the dataset's variability and complexity [Vafeiadis T, Diamantaras K I,Sarigiannidis G, et al., 2015].Huang conducted an analysis based on real customer data from an Irish telecom company. He compared several algorithms including logistic regression, linear classifiers, naive Bayes, decision trees, multi-layer perceptron neural networks, support vector machines, and genetic algorithms. His study aimed to determine which of these methods was most effective at predicting customer churn, offering a broad perspective on the relative strengths of these diverse techniques in handling real-world data [Huang, B., Kechadi, M.T., Buckley, B., 2012].Lu focused on real data from a telecommunications company to research churn prediction. Utilizing the boosting algorithm to weight and segment customer groups, Lu developed an early warning model for churn and compared it to a standard logistic regression model. The results showed that the boosted model provided superior predictive accuracy, illustrating the effectiveness of ensemble methods in enhancing churn prediction [Lu N, Lin H, Lu J, et al.,2014].Bi addressed the challenges of predicting customer churn in big data environments. He proposed a clustering method named SDSCM, which is based on Sequential Clustering Method (SCM) and Adaptive Feature Selection (AFS). This method was applied specifically to tackle churn in China Telecom's vast customer data, providing new strategies for improving customer churn management at scale [Bi, W., Cai, M., Liu, M., et al., 2016].Amin explored the use of rough set theory in the telecom industry to predict customer churn. He tested several algorithms, including exhaustive algorithms, genetic algorithms, and covering algorithms based on rough set theory. His work verified the effectiveness of rough set theory as a valuable tool in understanding and predicting customer churn, offering a novel approach to mining complex customer data[Amin, A., Anwar, S., Adnan, A., et al., 2017]. Although there is a large amount of academic research literature in the area, comprehensive study of the modeling of bank customer churn has not been pursued. The above characteristic data have been considered in the paper. This is used for a hybrid feature selection strategy with comprehensive sampling to further evaluate and have practical application value for the prediction of customer churning research in banks.
2.3 Review of customer churn influencing factors and
The present literature almost, in a quantitative manner, purely reviewed the customer churn-related factors from the perspective of customer characteristics, aiding, from many perspectives, banks in developing targeted programs and exploring marketing countermeasures for the key customer segments [Oskarsdottir, M., Baesens, B., Vanthienen, J., 2018]. And it wants to set its customer within the telecom and banking saturated industries that relatively have a fast-growing market and take into consideration the changes in value of the customer's lifecycle and the heterogeneity in Customer Lifetime Value (CLV). The proposed Expected Maximum Profit (EMP) metric measures with the characteristics of the customer base offer an insight into new customer retention [Kumar, A., Luthra, S., 2017].Use hierarchical analysis and the decision evaluation laboratory to propose a customer retention strategy in the automotive industry [Bahri-Ammari, N., Bilgihan, A., 2017].A telecommunication industry environment will believe that satisfaction and loyalty are the cardinal factors in predicting the operator's and customer retention as key predictors of satisfactory relationships. Distributive justice enhances customer satisfaction and loyalty [Bahri-Ammari, N., Bilgihan, A., 2017].
Distributive justice, with relationship-centered theorists such as Gurit, E.G., and Interactional Justice, suggests a link to customer loyalty and customer satisfaction in the telecommunication industry environment as one of the key determinants of operator and customer retention and satisfaction. [Diaz, G.R., 2017].Proposed that service quality has a two-sided positive and negative effect on customer satisfaction, using logistic regression and GSEM estimation methods. These were the only assessments showing a positive impact on the categories of customer satisfaction and also proving to have a statistically significant impact on other attributes of service, such as customer care, tariff and plan information, and billing clarity. Similar asymmetric results were found for other economic, socio-economic, and geographic determinants of customer decision making.
2.4 Shortcomings of Existing Research
Through the current literature research, I identified that most of the literature research on customer churn focuses on the use of big data mining technology for establishment through the data mining technology of the customer churn prediction model. And then, through the data to observe the behavior of customers and how to explore deeply the problem of the existence of massive raw data, category imbalance, and high feature dimension, development of a variety of intelligent algorithms and technologies to solve the problem, using it as a basis to verify the accuracy of the early-warning churn model, giving a comparison of good and bad models. In most of the literature studies, the causes of customer churn were observed separate from churn prediction and customer retention. In addition, while a few models are highly accurate for customer churn prediction, most prediction models are derived only from the data interaction between a customer and the enterprise. The blind data mining only considers the customer's own factors and ignores the outside environment that may cause aspects of customer churn easily hidden by the bank. The factors that lead to customer churn in the bank and the external environment are easily concealed. About whether a customer is lost or not, the research should dwell on a customer churn early warning that should not only simply predict whether the customer is lost but rather suggest how many of the relative customers it is worthy to retain and strategies for the same.Meanwhile, in the relatively mature field of imbalance data research in medicine, finance, and the internet, banks in this area of research are somewhat comprehensive. Further, discussion of customer enhancement strategies of banks generally focuses on maintaining and marketing key customers, and thus the bias in research objects. Also, the research studies the assessment of the space of profitability, the competitiveness cross-regional development, and market positioning, risk management, and capital replenishment in brief, but very little in terms of giving research on the early warning of bank customer loss risk. The paper thus takes the detailed analysis of the comprehensive consideration of the influencing factors of customer churn and elaborates on the current situation of bank development. After the model for early warning of customer churn risk is selected appropriately, the original data is processed so as to provide a good and predictive method for the customer relationship management of a bank, and then formulate the corresponding retention method based on research data results.
3. Project Requirements and Methodology
3.1Project requirements/sequence/classification
Table 3. 1 Project Requirements
A decision tree, which looks like a tree in its analytical model. The root node of the tree corresponds to the space of the data collection; each node of a branch corresponds to a classification problem, which is a test of a single attribute that results from splitting the space for data collection into two or more subsets; and each leaf node is a data partition with a categorization. The path from the root node of the decision tree to leaf nodes forms a prediction on the class of the corresponding object. The generation of the decision tree is a top-down, divide-and-conquer process that may be applied to a classification problem or a rule learning problem.The internal nodes of a decision tree represent attributes or sets of attributes, and the leaf nodes represent the categories or conclusions of the learning division. The features of the internal nodes are sometimes also referred to as test attributes or split attributes. The terminal nodes are categories; the branches that move down from the non-terminal nodes are the attribute values of the attributes. For a new instance that does not know the category, one can start from the root node of the decision tree and test for similar attributes, compare the attribute values corresponding to this node, then move down the branch, repeating the comparison and selecting the correct branch at each sub-node that it passes, until he reaches the leaf node where the class attributes of the sample instance await.
3.2.1 Decision Tree Construction Ideas
1. Starting with a training set of processed data and an empty tree, it checks each point of the current knot and divides.
2. If all training samples at the current node belong to the same class, then create a leaf node with the class label and finish. Labeled by that class, form a leaf node with that label and finish.
3. Compute every possible partition for each set using the optimal measure.
4. Select the optimal division as the test for the current node and create a child node of that division.
5. Name the edge of the parent and child classes with the output of that division, and divide the training data into the child node using the output of that division.
6. Treated as the child node, repeat procedures 2 to 5 in a loop with this as the new current node until there is no more divisible node for it.
3.2.2 Decision Tree Growth
The root node represents the whole dataset, and the tree develops with the recursive division of data to the subsets of nodes, where the division is carried out by testing all the given attributes until higher "purity" is achieved in that subset. This would basically mean that the purity of the samples taken in the branch is so high that it stops making sense for such samples to be further divided. It stops growing when all the samples have cohorts too small to divide further in a meaningful way. The main question of growing a decision tree is to establish the division criteria. This is to mean the diagram is able to show the growth process of a decision tree.
Figure 3. 1 Decision Tree Pruning
3.2.3 Pruning of decision tree
According to the growth process above for generating a decision tree, usually, interference outliers end up being the problem of overfitting, so that the new samples which are to be predicted by the decision tree categorized often do not make better results. So, there appeared a solution for avoiding such a problem: one should use the method of decision tree pruning. Commonly, the methods being used for pruning the decision tree are pre-pruning and post-pruning.
Pre-pruning can directly give the maximum depth in advance to avoid the minimum sample size of each tree node of the decision tree being too small on a certain tree node. All these prevent the tree from growing to its full depth, although setting an appropriate depth for the tree or sample threshold requires iterative trials in order to tune these parameters. This relies on the user having a fuller knowledge of the distribution patterns of the variables.
The post-pruning technique is waiting for the perfect growth of the decision tree, then it prunes off those branches that are of little significance with regards to classification based on certain criteria. It is basically a testing and pruning process. It calculates the prediction accuracy for the output variables of the current decision tree. It is supposed to give the maximum error rate which can be tolerated in advance; if the same is reached, then pruning is to be ended immediately; if not, pruning should continue. Post-pruning uses the test sample set's data for making a decision on the stopping point of the pruning process. It should stop the process of pruning until the error rate from the test set becomes drastically larger.
Figure 3.2 Construction of decision trees
3.2.4 Dataset Overview
Dataset Summary:
Entries: 175,028.
Features: 25 columns.
Description of Features:
Surname: Encoded identifier for customers.
CreditScore: Numerical score representing the creditworthiness of the customer.
Age: Customer's age.
Tenure: Number of years the customer has been with the bank.
Balance: Current account balance.
NumOfProducts: Number of bank products the customer uses.
HasCrCard: Indicates the possession of a bank-issued credit card (1 = Yes, 0 = No).
IsActiveMember: Active membership status (1 = Yes, 0 = No).
EstimatedSalary: Estimated annual salary of the customer.
Exited: Churn status (1 = Churned, 0 = Not churned).
Geographical and Demographic Data:
France, Germany, Spain: Binary indicators for the customer's country. Female, Male: Binary gender indicators.
Engineered Features:
Features derived from existing data such as Surname_tfidf_* (representing transformed surname features), various ratios or interactive features like Mem no Products, Cred_Bal_Sal, Bal_sal, Tenure_Age.
Preparation for Analysis:
The dataset may require preprocessing steps such as scaling, normalization, or encoding prior to modeling. Decision trees will benefit from an examination of feature importance to refine the model and enhance predictive accuracy.
Dataset Source:
The dataset can be accessed and downloaded from:
https://www.kaggle.com/datasets/cybersimar08/binary-classification-of-bank-c hurn-synthetic-data
3.2.5 Common Decision Tree Algorithms
3.2.5.1 CART Algorithm
Some other decision tree algorithms before CART were applied, among which is the famous ID3 and its improved version, C4.5. Applications of these algorithms were specifically classification tasks. The decision tree has some very unique aspects, as the application of the decision tree could be used not only for classification but also for regression problems. And with it, one chooses the classification attribute in the CART decision tree using the Gini index. The purity of the dataset D can be measured by the Gini value using the same notation as in Eq:
(3.1)
The smaller the Gini (D), the larger the importance of the purity of the dataset. Under the same notation as --, the Gini index for attribute a is defined as:
(3.2)
Thus, the following condition is to be satisfied by the selected attribute from the candidate attributes A for the minimum computation of the Gini index after making the segmentation with it.
(3.3)
3.2.5.2 Algorithm properties:
1. Dual purpose: It may be used in problems of classification (construction of classification trees) or regression problems (construction of regression trees).
2. Binary tree structure: CART is deterministic and always produces a binary tree, i.e., it always divides into two children in every node, while ID3, C4.5, and some other algorithms can but need not make multi-nomial trees.
3. Pruning technique: CART uses smart pruning that reduces the danger of overfitting and hence improves model generalization. Pruning is done by eliminating unwanted nodes from a fully developed tree that yield little influence on the predictive performance of the model.
3.2.5.3 Algorithmic Process:
1. Segmentation Criteria:In the classification, it divides the criterion of Gini impurity, and in the regression, it uses the least squares deviation.
2. Recursive Segmentation: Starting from the root node, the algorithm will generate, in a recursive way, two subsets from the current node's dataset while optimally choosing the cutting (splitting) point that minimizes the impurity (for classification) or bias (for regression) from the cut.
3. Stop condition: The other main benefits of CART are its high processing flexibility (could address different data and problems), the feature of no need for data pre-processing, and high interpretability. Furthermore, it can handle missing data and feature selection automatically in-built.