Introduction to Tensorflow

TensorFlow is an open-source numerical calculation for the implementation of machine learning models developed by the Google Brain team. Its architecture allows us to use machine learning and deep learning models with the support of CPU, GPU, TPU, and other distributed machines. It supports many programming languages such as C++, python software development etc. This blog aims to make you aware of the powerful API of Tensorflow i.e., High-level API. Tensorflow consists of multiple layer API let’s, check the below diagram.

IIn this tutorial, we are mainly going to cover the estimators in TensorFlow.

Prerequisites

• Install Tensorflow
• If using Anaconda Environment for Data Science, activate Tensorflow

Estimators

Tensorflow is used to build machine learning models. Using TensorFlow, we have to implement a model from scratch. But using estimators, we can also use built-in models like scikit-learn instead of bogging into the details. There are premade estimators, and custom estimators if you wish to build a model of your choice. All estimators are built under the tf.estimator.Estimator class. It involves the following actions:

• Training
• Evaluation
• Prediction
• Export for serving

How to write a program for premade estimators?

For the demo of these estimators, we will use the data of “Manthan Guess”. In this case study, we have to guess whether this person will churn out this bank, i.e., stop using the current account with the associated (single bank in this case) bank. Let’s check the steps.

Step 1: Writing an input function – A function for importing training and testing data

Do we have to pass data in every machine learning for model training or model evaluation? Similarly, in the Estimator API, we have to pass the input data (function) to input_fn (a standard function defined within TensorFlow). Here I will try to show how I define the input function for training.

#define train input_function

1. train_input_fn = tf.estimator.inputs.numpy_input_fn( 2. x={"CreditScore": np.array(data.CreditScore), 3. "Balance": np.array(data.Balance)}, 4. y=np.array(data['Exited']), 5. num_epochs=None, 6. shuffle=True)

Here, x represents the input-form, you can pass images, and multiple features here, we passed two inputs – “CreditScore”, “Balance”. Y represents output.

Step 2: Define a feature column – Identifying features for the model

At this step, we are trying to define the feature column to train the model. While building the model, we need features on which model will be trained.

1. feature_1 = tf.feature_column.numeric_column("CreditScore") 2. feature_2 = tf.feature_column.numeric_column("Balance") 3. feature_columns = [feature_1,feature_2]

In the above code, the first two lines are converted in form of features. And they were then appending it into feature columns for the training model.

Step 3: Initializing pre-made estimator

At this step, we will define a model to be used for solving the use case. There are two ways, the first is to use Premade Estimator, and the second is a custom estimator. We will use premade estimators of TensorFlow but, if you want, you can define a custom model as well. Let’s see how to use premade estimator for this dataset.

1. model = tf.estimator.LinearClassifier(feature_columns=feature_columns) 2. model.train(input_fn= train_input_fn, steps=2000)

Here, we used the LinearClassifier model for simplicity, but we can also use DNNClassifier, etc., available in the premade estimator.

The output of training is something like this:

INFO:tensorflow:Calling model_fn. INFO:tensorflow:Done calling model_fn. INFO:tensorflow:Create CheckpointSaverHook. INFO:tensorflow:Graph was finalized. INFO:tensorflow:Running local_init_op. INFO:tensorflow:Done running local_init_op. INFO:tensorflow:Saving checkpoints for 1 into C:\Users\gargr\AppData\Local\Temp\tmphl17weex\model.ckpt. INFO:tensorflow:loss = 88.72288, step = 1 INFO:tensorflow:global_step/sec: 225.266 INFO:tensorflow:loss = 23414.627, step = 101 (0.444 sec) INFO:tensorflow:global_step/sec: 478.944 INFO:tensorflow:loss = 12692.912, step = 201 (0.227 sec) INFO:tensorflow:global_step/sec: 478.156 INFO:tensorflow:loss = 29577.074, step = 301 (0.191 sec)

Step 4: Evaluating the model/Prediction for new values

The last step for model building is to evaluate it and predict new values. Let’s, see how to do it for our case study of churn prediction.

Here, creating a prediction input function (predict_input_fn)

1. test_data = data.loc[56:67,['CreditScore','Balance','Exited']] 2. test_data.reset_index(drop=True) 3. 4. expected = np.array(test_data.Exited) 5. expected 6. 7. predict_input_fn = tf.estimator.inputs.numpy_input_fn( 8. x={"CreditScore": np.array(test_data['CreditScore']), 9. "Balance": np.array(test_data['Balance'])}, 10. num_epochs=1, 11. shuffle=False) 12. 13. predictions = model.predict(input_fn=predict_input_fn)

In the above code, we defined the input prediction function for inputting data for prediction and used the trained model for predictions.

It is all about how to use premade estimators in Tensorflow.

If someone wants to write a custom estimate, they will follow the same steps instead of using the estimates already made. Custom estimators will write their model function. The model function will be written with the use of tf.layers.

So, this is all about how to use the estimator API.

Let’s check out the full code for this case study:

import required packages

import tensorflow as tf import numpy as np import matplotlib.pyplot as plt import pandas as pd

load the data

data = pd.read_csv('./Churn_Modelling.csv') data.head() msk = np.random.rand(len(data)) < 0.8 train=data[msk] test=data[~msk]

define training input function

#define train input_function train_input_fn = tf.estimator.inputs.numpy_input_fn( x={"CreditScore": np.array(train.CreditScore), "Balance": np.array(train.Balance)}, y=np.array(train['Exited']), num_epochs=None, shuffle=True) train_input_fn() [Output]: ({'Balance': <tf.Tensor 'random_shuffle_queue_DequeueMany_3:1' shape=(128,) dtype=float64> , 'CreditScore': <tf.Tensor 'random_shuffle_queue_DequeueMany_3:2' shape=(128,) dtype=int64> }, <tf.Tensor 'random_shuffle_queue_DequeueMany_3:3' shape=(128,) dtype=int64>)

Premade/ Canned Estimator defining

feature_1 = tf.feature_column.numeric_column("CreditScore") feature_2 = tf.feature_column.numeric_column("Balance") feature_columns = [feature_1,feature_2] model = tf.estimator.LinearClassifier(feature_columns=feature_columns, ) [Output]: INFO:tensorflow:Using default config. WARNING:tensorflow:Using temporary folder as model directory: C:\Users\gargr\AppData\Local\Temp\tmpci0vq1od INFO:tensorflow:Using config:{'_task_type': 'worker', '_global_id_in_cluster': 0, '_save_checkpoints_steps': None, '_num_ps_replicas': 0, '_save_checkpoints_secs': 600, '_train_distribute': None, '_tf_random_seed': None, '_cluster_spec': <tensorflow.python.training.server_lib.ClusterSpec object at 0x0000028327B0B390> , '_log_step_count_steps': 100, '_evaluation_master': '', '_is_chief': True, '_session_config': None, '_save_summary_steps': 100, '_service': None, '_num_worker_replicas': 1, '_model_dir': 'C:\\Users\\gargr\\AppData\\Local\\Temp\\tmpci0vq1od', '_keep_checkpoint_max': 5, '_task_id': 0, '_keep_checkpoint_every_n_hours': 10000, '_master': ''}

train the model

model.train( input_fn= train_input_fn, steps=2000) {Output]: INFO:tensorflow:Calling model_fn. INFO:tensorflow:Done calling model_fn. INFO:tensorflow:Create CheckpointSaverHook. INFO:tensorflow:Graph was finalized. INFO:tensorflow:Running local_init_op. INFO:tensorflow:Done running local_init_op. INFO:tensorflow:Saving checkpoints for 1 into C:\Users\gargr\AppData\Local\Temp\tmpci0vq1od\model.ckpt. INFO:tensorflow:loss = 88.72288, step = 1 INFO:tensorflow:global_step/sec: 189.175 INFO:tensorflow:loss = 46414.156, step = 101 (0.545 sec) INFO:tensorflow:global_step/sec: 455.523 INFO:tensorflow:loss = 58252.188, step = 201 (0.222 sec) INFO:tensorflow:global_step/sec: 509.2 INFO:tensorflow:loss = 12370.604, step = 301 (0.178 sec) INFO:tensorflow:global_step/sec: 549.406 INFO:tensorflow:loss = 16219.006, step = 401 (0.200 sec) INFO:tensorflow:global_step/sec: 508.595 INFO:tensorflow:loss = 9650.512, step = 501 (0.179 sec) INFO:tensorflow:global_step/sec: 502.775 INFO:tensorflow:loss = 3101.44, step = 601 (0.199 sec) INFO:tensorflow:global_step/sec: 540.225 INFO:tensorflow:loss = 7769.735, step = 701 (0.185 sec) INFO:tensorflow:global_step/sec: 524.367 INFO:tensorflow:loss = 36031.816, step = 801 (0.191 sec) INFO:tensorflow:global_step/sec: 547.759 INFO:tensorflow:loss = 1731.166, step = 901 (0.187 sec) INFO:tensorflow:global_step/sec: 482.969 INFO:tensorflow:loss = 23063.988, step = 1001 (0.204 sec) INFO:tensorflow:global_step/sec: 673.966 INFO:tensorflow:loss = 16607.012, step = 1101 (0.149 sec) INFO:tensorflow:global_step/sec: 560.367 INFO:tensorflow:loss = 18532.783, step = 1201 (0.176 sec) INFO:tensorflow:global_step/sec: 498.929 INFO:tensorflow:loss = 4076.1106, step = 1301 (0.205 sec) INFO:tensorflow:global_step/sec: 546.074 INFO:tensorflow:loss = 16558.512, step = 1401 (0.179 sec) INFO:tensorflow:global_step/sec: 519.489 INFO:tensorflow:loss = 4016.436, step = 1501 (0.192 sec) INFO:tensorflow:global_step/sec: 530.332 INFO:tensorflow:loss = 12311.674, step = 1601 (0.189 sec) INFO:tensorflow:global_step/sec: 541.875 INFO:tensorflow:loss = 19032.355, step = 1701 (0.185 sec) INFO:tensorflow:global_step/sec: 499.178 INFO:tensorflow:loss = 15493.949, step = 1801 (0.200 sec) INFO:tensorflow:global_step/sec: 568.69 INFO:tensorflow:loss = 14680.697, step = 1901 (0.176 sec) INFO:tensorflow:Saving checkpoints for 2000 into C:\Users\gargr\AppData\Local\Temp\tmpci0vq1od\model.ckpt. INFO:tensorflow:Loss for final step: 17477.594. <tensorflow.python.estimator.canned.linear.LinearClassifier at 0x283264f3828>

Make predictions

test_data = test.loc[:,['CreditScore','Balance','Exited']] test_data.reset_index(drop=True) expected = np.array(test_data.Exited) expected [Output]: array([0, 0, 0, ..., 0, 1, 0], dtype=int64) predict_input_fn = tf.estimator.inputs.numpy_input_fn( x={"CreditScore": np.array(test_data['CreditScore']), "Balance": np.array(test_data['Balance'])}, num_epochs=1, shuffle=False) predictions = model.predict(input_fn=predict_input_fn) predictions {Output]: <generator object Estimator.predict at 0x0000028325D2B990> for pred_dict, expect in zip(predictions,expected): class_id = pred_dict['class_ids'][0] probability = pred_dict['probabilities'][class_id] print("Prediction is {} with probability {} and expected is {}".format(class_id, 100 *probability,expect)) [Output]: INFO:tensorflow:Calling model_fn. INFO:tensorflow:Done calling model_fn. INFO:tensorflow:Graph was finalized. INFO:tensorflow:Restoring parameters from C:\Users\gargr\AppData\Local\Temp\tmpci0vq1od\model.ckpt-2000 INFO:tensorflow:Running local_init_op. INFO:tensorflow:Done running local_init_op. Prediction is 0 with probability 85.02780199050903 and expected is 0 Prediction is 0 with probability 79.51386570930481 and expected is 0 Prediction is 0 with probability 81.03984594345093 and expected is 0 Prediction is 0 with probability 89.28243517875671 and expected is 0 Prediction is 0 with probability 100.0 and expected is 0 Prediction is 0 with probability 100.0 and expected is 1 Prediction is 0 with probability 87.68146634101868 and expected is 0 Prediction is 0 with probability 100.0 and expected is 1 Prediction is 0 with probability 100.0 and expected is 0 Prediction is 0 with probability 100.0 and expected is 0 Prediction is 0 with probability 100.0 and expected is 0 Prediction is 0 with probability 100.0 and expected is 0 Prediction is 0 with probability 78.24604511260986 and expected is 0

I didn’t show full predictions here.

It is the full demo for estimators (pre-made). For accessing data of Churn prediction go to:

https://www.superdatascience.com/pages/deep-learning. If you will go to this link, go to part-1 and you will be able to download the dataset.

Conclusion

In this blog, we discussed Tensorflow and how to use high-level TensorFlow API. Tensorflow has a full stack of interfaces like low-level, mid-level, etc. In most of the parts, we see how to use estimators and a quick demo with the dataset.