### GCP with Tensorboard for Softmax Visualization

#### by allenlu2007

Excellent Reference:

https://github.com/aymericdamien/TensorFlow-Examples/

前文 (Tensorflow ABC) 提到 TensorFlow 的兩個好處是: (1) 提供 visualization tool (Tensor Board) 對於 debug 和理解 neural network behavior 非常重要。(2) 提供 google cloud platform (GCP) with built-in tensorflow software and tools, 可以做 large scale training and testing. 同樣非常重要。

本文主要討論 tensorboard 的使用方法，特別是在 GCP 上使用。

先說結論：在 caffe 的年代，TensorBoard 可能是唯一的 machine learning/deep learning visualization 工具。但目前有很多的 visualization tools. 例如 matlab, jupyter notebook+python plotting tools (matplotlib, pandas, etc.), 還有特別針對 deep learning visualization tool, 例如 deep dream.

Step 1: 在 GCP cloud shell 上用 python 執行 tensorflow. 同樣用 softmax classifer 作為例子。

tb.py (tensorboard example) code 如 hyperlink 或附件。

$ python tb.py

Step 2: 在 GCP cloud shell 執行 tensorboard. 注意要改 port from 6006 -> 8080.

$ tensorboard –logdir=/tmp/tensorflow_logs –port=8080

Launch Preview on port 8080!!

————————————————————————————–

''' Graph and Loss visualization using Tensorboard. This example is using the MNIST database of handwritten digits (http://yann.lecun.com/exdb/mnist/) Author: Aymeric Damien Project: https://github.com/aymericdamien/TensorFlow-Examples/ ''' from __future__ import print_function import tensorflow as tf # Import MNIST data from tensorflow.examples.tutorials.mnist import input_data mnist = input_data.read_data_sets("/tmp/data/", one_hot=True) # Parameters learning_rate = 0.01 training_epochs = 25 batch_size = 100 display_step = 1 logs_path = '/tmp/tensorflow_logs/example' # tf Graph Input # mnist data image of shape 28*28=784 x = tf.placeholder(tf.float32, [None, 784], name='InputData') # 0-9 digits recognition => 10 classes y = tf.placeholder(tf.float32, [None, 10], name='LabelData') # Set model weights W = tf.Variable(tf.zeros([784, 10]), name='Weights') b = tf.Variable(tf.zeros([10]), name='Bias') # Construct model and encapsulating all ops into scopes, making # Tensorboard's Graph visualization more convenient with tf.name_scope('Model'): # Model pred = tf.nn.softmax(tf.matmul(x, W) + b) # Softmax with tf.name_scope('Loss'): # Minimize error using cross entropy cost = tf.reduce_mean(-tf.reduce_sum(y*tf.log(pred), reduction_indices=1)) with tf.name_scope('SGD'): # Gradient Descent optimizer = tf.train.GradientDescentOptimizer(learning_rate).minimize(cost) with tf.name_scope('Accuracy'): # Accuracy acc = tf.equal(tf.argmax(pred, 1), tf.argmax(y, 1)) acc = tf.reduce_mean(tf.cast(acc, tf.float32)) # Initializing the variables init = tf.global_variables_initializer() # Create a summary to monitor cost tensor tf.summary.scalar("loss", cost) # Create a summary to monitor accuracy tensor tf.summary.scalar("accuracy", acc) # Merge all summaries into a single op merged_summary_op = tf.summary.merge_all() # Launch the graph with tf.Session() as sess: sess.run(init) # op to write logs to Tensorboard summary_writer = tf.summary.FileWriter(logs_path, graph=tf.get_default_graph()) # Training cycle for epoch in range(training_epochs): avg_cost = 0. total_batch = int(mnist.train.num_examples/batch_size) # Loop over all batches for i in range(total_batch): batch_xs, batch_ys = mnist.train.next_batch(batch_size) # Run optimization op (backprop), cost op (to get loss value) # and summary nodes _, c, summary = sess.run([optimizer, cost, merged_summary_op], feed_dict={x: batch_xs, y: batch_ys}) # Write logs at every iteration summary_writer.add_summary(summary, epoch * total_batch + i) # Compute average loss avg_cost += c / total_batch # Display logs per epoch step if (epoch+1) % display_step == 0: print("Epoch:", '%04d' % (epoch+1), "cost=", "{:.9f}".format(avg_cost)) print("Optimization Finished!") # Test model # Calculate accuracy print("Accuracy:", acc.eval({x: mnist.test.images, y: mnist.test.labels})) print("Run the command line:\n" \ "--> tensorboard --logdir=/tmp/tensorflow_logs " \ "\nThen open http://0.0.0.0:6006/ into your web browser")

---------------------------------------------------