10.2. Exploring Our Data Set

In this section, we will learn how to analyze an enormous data set to find interesting associations between its various items. We will also learn how to evaluate the data set’s different behaviors to make predictions about its future behaviors. For this exercise, we will use shopping data from Instacart.

Instacart is a shopping and delivery service that works with stores in your city such as Whole Foods, Costco, and local stores. They pick up your order and deliver it to your door. Consequently, they have a lot of data on a lot of different shopping behaviors that we can use to make predictions about future purchases or suggest items that a person may want to add to their shopping cart based on their past behavior. Sound familiar? This is the kind of thing that Amazon has been doing successfully for years.

%matplotlib inline

import pandas as pd
import matplotlib
import matplotlib.pyplot as plt
import numpy as np
import seaborn as sbn
import altair as alt
import requests
matplotlib.style.use('ggplot')
sbn.set_style("whitegrid")
import json
import pickle
import scipy
alt.data_transformers.enable('json')
DataTransformerRegistry.enable('json')

10.2.2. The Data

The data we will be using in this module is shopping data from Instacart, the Instacart Online Grocery Shopping Dataset 2017, accessed from https://www.instacart.com/datasets/grocery-shopping-2017 on October 15, 2018. This is an enormous data set. (The largest file has over 32 million rows!) We don’t want to start with that file, as that is very big. So, we have a smaller file, still with 1.3 million rows, with which we can start.

10.2.3. aisles.csv

aisle_id,aisle
1,prepared soups salads
2,specialty cheeses
3,energy granola bars
...

10.2.4. departments.csv

department_id,department
1,frozen
2,other
3,bakery
...

10.2.5. order_products__prior.csv

These files specify which products were purchased in each order. order_products__prior.csv contains previous order contents for all customers. reordered indicates that the customer has a previous order that contains the product. Note that some orders will have no reordered items. order_products_train.csv is much smaller (even though it has 1.3 million records), and is a better place to start.

order_id,product_id,add_to_cart_order,reordered
1,49302,1,1
1,11109,2,1
1,10246,3,0
...

10.2.6. orders.csv

This file tells to which set (prior, train, test) an order belongs. You are predicting reordered items only for the test set orders. order_dow is the day of the week.

order_id,user_id,eval_set,order_number,order_dow,order_hour_of_day,days_since_prior_order
2539329,1,prior,1,2,08,
2398795,1,prior,2,3,07,15.0
473747,1,prior,3,3,12,21.0
...

10.2.7. products.csv

product_id,product_name,aisle_id,department_id
1,Chocolate Sandwich Cookies,61,19
2,All-Seasons Salt,104,13
3,Robust Golden Unsweetened Oolong Tea,94,7
...
You have attempted of activities on this page