TensorFlow is an open-source numerical calculation for the implementation of machine learning models developed by the Google Brain team. Its architecture allows us to use machine learning and deep learning models with the support of CPU, GPU, TPU, and other distributed machines. It supports many programming languages, such as C++, python, etc. Expert developers of a leading Python software development company have written this blog and aim to make you aware of the powerful API of Tensorflow i.e., High-level API. Tensorflow consists of multiple-layer API. Let’s, check the below diagram.
IIn this tutorial, we are mainly going to cover the estimators in TensorFlow.
Prerequisites
- Install Tensorflow
- If using Anaconda Environment for Data Science, activate Tensorflow
Estimators
Tensorflow is used to build machine learning models. Using TensorFlow, we have to implement a model from scratch. But using estimators, we can also use built-in models like scikit-learn instead of bogging into the details. There are premade estimators, and custom estimators if you wish to build a model of your choice. All estimators are built under the tf.estimator.Estimator class. It involves the following actions:
- Training
- Evaluation
- Prediction
- Export for serving
How to write a program for premade estimators?
For the demo of these estimators, we will use the data of “Manthan Guess”. In this case study, we have to guess whether this person will churn out this bank, i.e., stop using the current account with the associated (single bank in this case) bank. Let’s check the steps.
Step 1: Writing an input function – A function for importing training and testing data
Do we have to pass data in every machine learning for model training or model evaluation? Similarly, in the Estimator API, we have to pass the input data (function) to input_fn (a standard function defined within TensorFlow). Here I will try to show how I define the input function for training.
#define train input_function
1. train_input_fn = tf.estimator.inputs.numpy_input_fn(
2. x={"CreditScore": np.array(data.CreditScore),
3. "Balance": np.array(data.Balance)},
4. y=np.array(data['Exited']),
5. num_epochs=None,
6. shuffle=True)
Here, x represents the input-form, you can pass images, and multiple features here, we passed two inputs – “CreditScore”, “Balance”. Y represents output.
Step 2: Define a feature column – Identifying features for the model
At this step, we are trying to define the feature column to train the model. While building the model, we need features on which model will be trained.
1. feature_1 = tf.feature_column.numeric_column("CreditScore")
2. feature_2 = tf.feature_column.numeric_column("Balance")
3. feature_columns = [feature_1,feature_2]
In the above code, the first two lines are converted in form of features. And they were then appending it into feature columns for the training model.
Step 3: Initializing pre-made estimator
At this step, we will define a model to be used for solving the use case. There are two ways, the first is to use Premade Estimator, and the second is a custom estimator. We will use premade estimators of TensorFlow but, if you want, you can define a custom model as well. Let’s see how to use premade estimator for this dataset.
1. model = tf.estimator.LinearClassifier(feature_columns=feature_columns)
2. model.train(input_fn= train_input_fn, steps=2000)
Here, we used the LinearClassifier model for simplicity, but we can also use DNNClassifier, etc., available in the premade estimator.
The output of training is something like this:
INFO:tensorflow:Calling model_fn.
INFO:tensorflow:Done calling model_fn.
INFO:tensorflow:Create CheckpointSaverHook.
INFO:tensorflow:Graph was finalized.
INFO:tensorflow:Running local_init_op.
INFO:tensorflow:Done running local_init_op.
INFO:tensorflow:Saving checkpoints for 1 into C:\Users\gargr\AppData\Local\Temp\tmphl17weex\model.ckpt.
INFO:tensorflow:loss = 88.72288, step = 1
INFO:tensorflow:global_step/sec: 225.266
INFO:tensorflow:loss = 23414.627, step = 101 (0.444 sec)
INFO:tensorflow:global_step/sec: 478.944
INFO:tensorflow:loss = 12692.912, step = 201 (0.227 sec)
INFO:tensorflow:global_step/sec: 478.156
INFO:tensorflow:loss = 29577.074, step = 301 (0.191 sec)
Step 4: Evaluating the model/Prediction for new values
The last step for model building is to evaluate it and predict new values. Let’s, see how to do it for our case study of churn prediction.
Here, creating a prediction input function (predict_input_fn)
1. test_data = data.loc[56:67,['CreditScore','Balance','Exited']]
2. test_data.reset_index(drop=True)
3.
4. expected = np.array(test_data.Exited)
5. expected
6.
7. predict_input_fn = tf.estimator.inputs.numpy_input_fn(
8. x={"CreditScore": np.array(test_data['CreditScore']),
9. "Balance": np.array(test_data['Balance'])},
10. num_epochs=1,
11. shuffle=False)
12.
13. predictions = model.predict(input_fn=predict_input_fn)
In the above code, we defined the input prediction function for inputting data for prediction and used the trained model for predictions.
It is all about how to use premade estimators in Tensorflow.
If someone wants to write a custom estimate, they will follow the same steps instead of using the estimates already made. Custom estimators will write their model function. The model function will be written with the use of tf.layers.
So, this is all about how to use the estimator API.
Let’s check out the full code for this case study:
import required packages
import tensorflow as tf
import numpy as np
import matplotlib.pyplot as plt
import pandas as pd
load the data
data = pd.read_csv('./Churn_Modelling.csv')
data.head()
No. |
RowNumber |
CustomerId |
Surname |
CreditScore |
Geography |
Gender |
Age |
Tenure |
Balance |
NumOfProducts |
HasCrCard |
IsActiveMember |
EstimatedSalary |
Exited |
0 |
1 |
15634602 |
Hargrave |
619 |
France |
Female |
42 |
2 |
0.00 |
1 |
1 |
1 |
101348.88 |
1 |
1 |
2 |
15647311 |
Hill |
608 |
Spain |
Female |
41 |
1 |
83807.86 |
1 |
0 |
1 |
112542.58 |
0 |
2 |
3 |
15619304 |
Onio |
502 |
France |
Female |
42 |
8 |
159660.80 |
3 |
1 |
0 |
113931.57 |
1 |
3 |
4 |
15701354 |
Boni |
699 |
France |
Female |
39 |
1 |
0.00 |
2 |
0 |
0 |
93826.63 |
0 |
4 |
5 |
15737888 |
Mitchell |
850 |
Spain |
Female |
43 |
2 |
125510.82 |
1 |
1 |
1 |
79084.10 |
0 |
msk = np.random.rand(len(data)) < 0.8 train=data[msk] test=data[~msk] define training input function
#define train input_function
train_input_fn = tf.estimator.inputs.numpy_input_fn(
x={"CreditScore": np.array(train.CreditScore),
"Balance": np.array(train.Balance)},
y=np.array(train['Exited']),
num_epochs=None,
shuffle=True)
train_input_fn()
[Output]:
({'Balance':
,
'CreditScore':
},
)
Premade/ Canned Estimator defining
feature_1 = tf.feature_column.numeric_column("CreditScore")
feature_2 = tf.feature_column.numeric_column("Balance")
feature_columns = [feature_1,feature_2]
model = tf.estimator.LinearClassifier(feature_columns=feature_columns, )
[Output]:
INFO:tensorflow:Using default config.
WARNING:tensorflow:Using temporary folder as model directory:
C:\Users\gargr\AppData\Local\Temp\tmpci0vq1od
INFO:tensorflow:Using config:{'_task_type': 'worker', '_global_id_in_cluster': 0,
'_save_checkpoints_steps': None, '_num_ps_replicas': 0, '_save_checkpoints_secs': 600,
'_train_distribute': None, '_tf_random_seed': None, '_cluster_spec':
,
'_log_step_count_steps': 100, '_evaluation_master': '', '_is_chief': True, '_session_config':
None, '_save_summary_steps': 100, '_service': None, '_num_worker_replicas': 1,
'_model_dir': 'C:\\Users\\gargr\\AppData\\Local\\Temp\\tmpci0vq1od', '_keep_checkpoint_max': 5,
'_task_id': 0, '_keep_checkpoint_every_n_hours': 10000, '_master': ''}
train the model
model.train(
input_fn= train_input_fn, steps=2000)
{Output]:
INFO:tensorflow:Calling model_fn.
INFO:tensorflow:Done calling model_fn.
INFO:tensorflow:Create CheckpointSaverHook.
INFO:tensorflow:Graph was finalized.
INFO:tensorflow:Running local_init_op.
INFO:tensorflow:Done running local_init_op.
INFO:tensorflow:Saving checkpoints for 1 into C:\Users\gargr\AppData\Local\Temp\tmpci0vq1od\model.ckpt.
INFO:tensorflow:loss = 88.72288, step = 1
INFO:tensorflow:global_step/sec: 189.175
INFO:tensorflow:loss = 46414.156, step = 101 (0.545 sec)
INFO:tensorflow:global_step/sec: 455.523
INFO:tensorflow:loss = 58252.188, step = 201 (0.222 sec)
INFO:tensorflow:global_step/sec: 509.2
INFO:tensorflow:loss = 12370.604, step = 301 (0.178 sec)
INFO:tensorflow:global_step/sec: 549.406
INFO:tensorflow:loss = 16219.006, step = 401 (0.200 sec)
INFO:tensorflow:global_step/sec: 508.595
INFO:tensorflow:loss = 9650.512, step = 501 (0.179 sec)
INFO:tensorflow:global_step/sec: 502.775
INFO:tensorflow:loss = 3101.44, step = 601 (0.199 sec)
INFO:tensorflow:global_step/sec: 540.225
INFO:tensorflow:loss = 7769.735, step = 701 (0.185 sec)
INFO:tensorflow:global_step/sec: 524.367
INFO:tensorflow:loss = 36031.816, step = 801 (0.191 sec)
INFO:tensorflow:global_step/sec: 547.759
INFO:tensorflow:loss = 1731.166, step = 901 (0.187 sec)
INFO:tensorflow:global_step/sec: 482.969
INFO:tensorflow:loss = 23063.988, step = 1001 (0.204 sec)
INFO:tensorflow:global_step/sec: 673.966
INFO:tensorflow:loss = 16607.012, step = 1101 (0.149 sec)
INFO:tensorflow:global_step/sec: 560.367
INFO:tensorflow:loss = 18532.783, step = 1201 (0.176 sec)
INFO:tensorflow:global_step/sec: 498.929
INFO:tensorflow:loss = 4076.1106, step = 1301 (0.205 sec)
INFO:tensorflow:global_step/sec: 546.074
INFO:tensorflow:loss = 16558.512, step = 1401 (0.179 sec)
INFO:tensorflow:global_step/sec: 519.489
INFO:tensorflow:loss = 4016.436, step = 1501 (0.192 sec)
INFO:tensorflow:global_step/sec: 530.332
INFO:tensorflow:loss = 12311.674, step = 1601 (0.189 sec)
INFO:tensorflow:global_step/sec: 541.875
INFO:tensorflow:loss = 19032.355, step = 1701 (0.185 sec)
INFO:tensorflow:global_step/sec: 499.178
INFO:tensorflow:loss = 15493.949, step = 1801 (0.200 sec)
INFO:tensorflow:global_step/sec: 568.69
INFO:tensorflow:loss = 14680.697, step = 1901 (0.176 sec)
INFO:tensorflow:Saving checkpoints for 2000 into C:\Users\gargr\AppData\Local\Temp\tmpci0vq1od\model.ckpt.
INFO:tensorflow:Loss for final step: 17477.594.
Make predictions
test_data = test.loc[:,['CreditScore','Balance','Exited']]
test_data.reset_index(drop=True)
expected = np.array(test_data.Exited)
expected
[Output]:
array([0, 0, 0, ..., 0, 1, 0], dtype=int64)
predict_input_fn = tf.estimator.inputs.numpy_input_fn(
x={"CreditScore": np.array(test_data['CreditScore']),
"Balance": np.array(test_data['Balance'])},
num_epochs=1,
shuffle=False)
predictions = model.predict(input_fn=predict_input_fn)
predictions
{Output]:
for pred_dict, expect in zip(predictions,expected):
class_id = pred_dict['class_ids'][0]
probability = pred_dict['probabilities'][class_id]
print("Prediction is {} with probability {} and expected is {}".format(class_id, 100 *probability,expect))
[Output]:
INFO:tensorflow:Calling model_fn.
INFO:tensorflow:Done calling model_fn.
INFO:tensorflow:Graph was finalized.
INFO:tensorflow:Restoring parameters from C:\Users\gargr\AppData\Local\Temp\tmpci0vq1od\model.ckpt-2000
INFO:tensorflow:Running local_init_op.
INFO:tensorflow:Done running local_init_op.
Prediction is 0 with probability 85.02780199050903 and expected is 0
Prediction is 0 with probability 79.51386570930481 and expected is 0
Prediction is 0 with probability 81.03984594345093 and expected is 0
Prediction is 0 with probability 89.28243517875671 and expected is 0
Prediction is 0 with probability 100.0 and expected is 0
Prediction is 0 with probability 100.0 and expected is 1
Prediction is 0 with probability 87.68146634101868 and expected is 0
Prediction is 0 with probability 100.0 and expected is 1
Prediction is 0 with probability 100.0 and expected is 0
Prediction is 0 with probability 100.0 and expected is 0
Prediction is 0 with probability 100.0 and expected is 0
Prediction is 0 with probability 100.0 and expected is 0
Prediction is 0 with probability 78.24604511260986 and expected is 0
I didn’t show full predictions here.
It is the full demo for estimators (pre-made). For accessing data of Churn prediction go to:
https://www.superdatascience.com/pages/deep-learning. If you will go to this link, go to part-1 and you will be able to download the dataset.
Conclusion
In this blog, we discussed Tensorflow and how to use high-level TensorFlow API. Tensorflow has a full stack of interfaces like low-level, mid-level, etc. In most of the parts, we see how to use estimators and a quick demo with the dataset.