Udacity Capstone | Analysing and Predicting Starbucks Customers' Reponses towards 10 kinds of Promotion Offers

Introduction

With the advent of the era of big data, companies are more and more inclined to analyze customer consumption behaviors, in order to formulate specific marketing strategies to promote consumers to complete the transactions.

Customer groups of different ages and different incomes obviously have different consumption habits. Taking age as an example, young people are more susceptible to online and social media advertisements. And more, because of their relatively less savings, they will be more concerned about the price and the promotion.
In this context, Starbucks has developed an experimental system that simulates user consumption data, and analyzes the data to find the patterns of customer consumption, so as to conduct more targeted promotions and optimize revenue.

Once every few days, Starbucks sends out an offer to users of the mobile app. An offer can be merely an advertisement for a drink or an actual offer such as a discount or BOGO (buy one get one free).
Some users might not receive any offer during certain weeks.
Not all users receive the same offer.

This data set contains simulated data that mimics customer behavior on the Starbucks rewards mobile app, it’s a simplified version of the real Starbucks app because the underlying simulator only has one product whereas Starbucks actually sells dozens of products.

The task is to combine transaction, demographic and offer data to determine which demographic groups respond best to which offer type.

More precisely, I aim to answer two questions:

1. What are the main factors that driver customers or groups to complete an offer?
2. Given offer characteristics and user demographics, can we predict whether the customer will complete the offer effectively? What’s more, how much money will the customer pay for?

Data Set

• portfolio.json

containing offer ids and meta data about each offer (duration, type, etc.)

• profile.json
  demographic data for each customer | Columns | Data Type | Explanation | Total Count | NaN Count | | —- | —- | —- | —- | —- | | age | int | the age of customer | 14825 | 2175 | | became_member_on | int | the enroll date of customer
  e.g. 20170101 | 17000 | - | | gender | str | the gender of customer
  values:‘male’,’female’,’other’ | 17000 | - | | id | str | the id of customer | 17000 | - | | income | float | the income of customer | 14825 | 2175 |

• transcript.json
  records for transactions, offers received, offers viewed, and offers completed. It shows user purchases made on the app including the timestamp of purchase and the amount of money spent on a purchase. | Columns | Data Type | Explanation | Total Count | NaN Count | | —- | —- | —- | —- | —- | | person | str | the id of customer | 306534 | - | | event | str | the description of transcript
  values:‘offer received’,’offer viewed’,’transaction’,’offer completed’ | 306534 | - | | time | int | the happend time of event(hour) | 306534 | - | | value | str dict | some includes id of offer, some includes amount of transaction | 306534 | - |

Data Cleaning and Preprocessing

At the begining, I try to know the basic informations about NaN values, they all bounded with the unusual age value “118”. After all these NaN records have been deleted, Here comes the distributions of gender, age and income.

Immediately after, I wrangled the data transcript by extracting the value column to offer id and amount columns. What’s more, I merged the ‘duration’ and ‘offer_type’ infos bounded with the offer id for further data preprocessing. Below is description of the wrangled ‘transcript’:

By using the data set above, based on a self designed program flow chart(File: data_preprocessing_class.py), I extract the transactions infomation direct to the related person and offer.

Meantime, I wrangle the data type of some features (e.g. the ‘value’ column in portfolio is a list, should be unfolded), and transform some features to normal form(e.g. transform the member enroll date from ‘int’ to ‘date’).

Then I divide all the customers into 12 segments according to ‘age’ and ‘income’, which tends to show a group characteristics.

What’s more important, considering all kinds of response situations to offer, I divide all transactions to 4 groups:

Finally, I get an ideal wrangled data set with label of segments and response groups described as follows(File: model_dataset_raw):

Notice 1.: label_effective_offer (Label describes the completed level of offer)
(Attention: there is no infomation about ‘offer viewed’)

Notice 2.: label_group (4 groups of resonse to offers)

Notice 3.: label_seg (12 segments based on age and income)

Income Level:

Notice 4.: offer_id (10 Kinds of offer)

Exploration analysis

Below is the structure of the analysis data:

The data set has been divided into 12 Segments based on age and income, and each segment has 4 response groups.
With different combination of the 4 response groups, there are 3 questions to be explored.

What’s more, I will use the index IIR to discuss, whether the offer is significantly popular by customer.

Q1 Offers Distribution

• In general

1. There are just 5 person, who never received the offer (offer_id is ‘-1’)
   1. Two in segment #7; One in each segment #8 #9 #11.
   2. They all are more than 50 years old and has more than one year membership. It seems they are regular customer and needn’t receive the offer.
2. The offer distributions under income: see segment #3 VS. segment #12
   1. Young people have not so much money.
   2. Elder people tend to have more savings.
3. The offer distributions under age: see segment #1 VS. segment #10
   1. In the low income group, compared with young person, the elder person seems to receive less offers

• In segments(subplots)

1. In each segment, person reveive almost the same quantity of offers. See the average line.

Q2 Interest Distribution towards different offers

• In general

1. The customers care more about offer 0, 1, 5, 6, 8
2. Offer 2, 3, 4 are not in interest
3. For offer 7, 9, some cares, some doesn’t care

From the three tables above, we can conclude that:

1. ‘social’ is an import factor to attract people to complete the offer
2. ‘bogo’ with medium difficulty are more popular
3. ‘discount’ with less difficulty are more popular

• In segments(subplots)

1. Offer 0, 1
   1. The high income groups show less interst compared with other income group. (see Segments #3 #6 #9 #12)
2. Offer 5, 6, 8
   1. Customer shows great interst(in all Segments)

Q3 Difficulty Distribution of different offers

• Summary: Level of completion for each offer

1. Offer 5, 6, 8 are better completed
   1. most are ‘discount’, and there is ‘social’ factor
   2. ‘difficulty’ of ‘bogo’ is not so much
2. Offer 0, 1 are harder to complete
   1. ‘difficulty’ of ‘bogo’ is a little bit heavy
3. Offer 7: an informational offer
   1. richer customers don’t care (see segment #6 #9 #12, compared to the average line)
   2. more attracted to less rich people (see average line)
4. Offer 2, 3, 4 are more attracted in medium elder and rich people (see segment #5 #8)
5. The person with high income tends to complete all offers (see segment #6 #9 #12)
   1. even for the offer 0, 1

Q4 Index IIR: is the offer significant popular?

Definition of IIR: Incremental Response Rate

• In general: which offer seems popular?

1. Offer 2, 3, 4 have positive IIR
   1. for offer 4, the difficulty is 20, maybe few people want to complete it, so it shows high IIR
   2. for offer 3, difficulty is not big, but reward is ok, so it’s popular
2. Offer 0,1 have a huge negtive IIR
   1. specially for the low income person(see segments #1 #4 #7 #10)

• In segments(subplots)

1. The rich people seem not so excited about offer received(see Segments #6 #9 #12)
   1. Maybe they are too rich to be encouraged from the reward of offer

Summary for Data Exploration

1. The channel to send an offer
   • Through ‘social’ is a better way.
     
2. The type of offer
   • People like ‘discount’, because the reward is real money compared to ‘bogo’(get another same thing) and ‘informational’(just information)
     
3. The content of offer
   • If the ‘difficulty’ is too much, e.g. 20, people have less desire to complete the offer
   • 5-10 maybe a good range of ‘difficulty’
     
4. For all offers, considering of Interest(care) and Difficulty(Level of completion)
   • Offer 5, 6, 8 are more attracted and more easy to complete: could be sent to all customers
   • Offer 0, 1 are hard to complete, but could be sent to high income customers
   • Offer 2, 3, 4 are more attracted to the medium elder, income people(Segment #5 #8)
   • Offer 7 for less income people(maybe the information is attracted by them)

Feature Extraction

Based on the model_dataset_raw above, after some objects columns(offer_id, offer_type, gender etc.) transformed to 0-1 variables, all the potential features have been extracted.

……
offer_9 | int | 0-1 variables of 10 kinds offers | 66501 | - |

Model

I wonder whether Machine Learning will find some intersting points of the data. Especially in the following situations:

1. Offer is going to be sent to a customer, will this offer effective?
2. Offer is already sent to a customer, is this offer effective?
3. Given basic infos of a customer, how to recommend an offer with the most effctivity?

To answer these questions, I build a model pipeline:

• Select features and target(for different issue concerned use different features and target)
• Select classifiers and compare the perform of all classifiers
• Select the suitable parameters of the best performed classifier by using grid search method
• Analyse the result

Notice:** One Neural Network is also built for regeression analysing.

Issue1: Offer is going to be sent to a customer, will this offer effective?

Model training result:

Issue1 Summary:

1. The first model KNeighborsClassifier could be a reference model
2. SVC and NuSVC take more time, I attempt to continue without them.
3. DecisionTreeClassifier and RandomForestClassifier both have a high score in training, is there something speciall?
   • Notice that: the test time is much more less than train time. The possible reason is that the data set has a simple structure so that all predict with same result(see the deployment of Issue2: no matter how I change the input data, it all shows the same result.)
4. For all, the accuracy of predicting is around 70%, it seems models are not so appropriate in this situation.

Issue2: Offer is already sent to a customer, is this offer effective?

Model training result:

Issue2 Summary:

1. As a reference model, KNeighborsClassifier performs not bad.
2. DecisionTreeClassifier and RandomForestClassifier both have a full score in training, is there something speciall?
   • Notice that: the test time is much more less than train time. The possible reason is that the data set has a simple structure so that all predict with same result(see the deployment of Issue2: no matter how I change the input data, it all shows the same result.)

P.S.: I’ve used the GradientBoostingClassifier as the target model to deploy my project.

Issue3: Given basic infos of a customer, how to recommend an offer with the most effctivity?

Model training result:

Issue3 Summary:

1. For all, the predicting performs are totally bad.
   • But still, DecisionTreeClassifier and RandomForestClassifier have high score in training, what’s going on?
     • Notice that: the test time if much more less than train time. The possible reason is that the data set has a simple structure so that all predict with same result(see the deployment of Issue2: no matter how I change the input data, it all shows the same result.)
2. Pay attention to that the GradientBoostingClassifier() is not suitable for multi-class problem.
   So I used MultiOutputClassifier(GradientBoostingClassifier())

Additional Issue: Neural Network for regeression

Model training result:

Additional Issue summary:

This experiment for regression analysing seems useless because of the stagnant and huge training loss.

Results: deployment of Issue2

Offer is already sent to a customer, is this offer effective?
Classifier: GradientBoostingClassifier
Features importances order:

Deployment - input the data:

Deployment - result:

Conclusion

1. Two analysis methods: heuristic exploration & model building
   1. The model method fits not good at the reality: for example the deployment of Issue2, when I change the input data, the result seems always the same.
   2. But the heuristic exploration makes more sense.
   3. In this case we could try unsupervised Machine Learning method like Cluster. See References[6]
2. About Data Set
   1. Maybe when the amount of data is big enough, we could get some resonable founds by using the supervised Machine Learning method.
   2. Besides, there is no features of customer id, the customer exists in the form of the segment group, maybe when the transactions of an unique customer more frequently occurs, there would be some patterns of the consuming behavior.
3. About the Segment
   1. Here I use the information of age and income. We can also use other method to segment the customers, e.g. age and gender.

For more details of this project, you could refer to my Github Repository
I would like to thank Udacity & Starbucks for all the supports, especially for teaching assistant.

References

[1]Create dummies from a column with multiple values in pandas
[2]Starbucks Capstone Challenge: Using Starbucks app user data to predict effective offers
[3]Starbucks Promotion Optimization
[4]generations-and-age
[5]single taxable income
[6]Investigating Starbucks Customers Segmentation using Unsupervised Machine Learning