Category Archives: GPGPU

Experiencing Deep Learning with Jupyter and Anaconda

Most of the time my work with deep learning is done in command line interface with Python and TensorFlow. The clean and efficient syntax of the Python language and package design of TensorFlow almost eliminated the need of a complex Integrated Development Environment (IDE). But after trying out the free Google Colab service that provide a web based interface in Jupyter, I am going to set up one on my desktop that sports an Nvidia RTX2060 GPU.

Installation is easy, but be sure to run Anaconda console as Administrator on Windows platform. For running TensorFlow with GPU:

conda create -n tensorflow_gpuenv tensorflow-gpu
conda activate tensorflow_gpuenv

Managing multiple packages is much easier with Anaconda as it separate configurations into environments that can be customized. On my development machine, I can simply create a TensorFlow environment with GPU and then install Jupyter to enjoy its graphical interface.

Finally to activate Jupyter:

jupyter notebook

jupyterconsole.PNG

To see how Anaconda with Jupyter is flexible on the same machine, a comparison of a simple image pattern recognition program runs under Jupyter with and without GPU support.

jupytergpu
jupytercpu

Machine learning with a classic game Doom and TensorFlow

Busy this weekend with work. During those irregular shift, I decided to let my GeForce RTX 2060 to play some game while I am away from home.

VizDoom is a port of the classic first person shooting game Doom to the machine learning area. With this specialised port compatible with Python, it is easy to train neural network to play this human game with TensorFlow. Over the weekend, my GPU card played countless episodes of games to learn how to win on itself while I am working in office.

doom2

 

Profiling machine learning applications in TensorFlow

TensorFlow provided package timeline by using the import from tensorflow.python.client

from tensorflow.python.client import timeline

This is useful for performance profiling TensorFlow application with graphical visualization similar to the graphs generated from the CUDA Visual Profiler. With a little tweak in the machine learning code, TensorFlow applications can store and report performance metrics of the learning process.
tfprofile3

The design of the timeline package made it easy to add profiling by simply adding code below.

run_options = tf.RunOptions(trace_level=tf.RunOptions.FULL_TRACE)
run_metadata = tf.RunMetadata() 

It is also required to instruct the model to compile with the profiling options:

model.compile(optimizer='adam',
loss='sparse_categorical_crossentropy',
metrics=['accuracy'],
options=run_options,
run_metadata=run_metadata)

With the sample mnist digits classifier for TensorFlow, the output shown Keras history are saved and can be later retrieved to generate reports.
tfprofile2

Finally, using the Chrome tracing page ( chrome://tracing/ ), the performance metrics persisted on file system can be opened for verification.
tfprofile1

 

TensorFlow and Keras on RTX2060 for pattern recognition

The MNIST database is a catalog of handwritten digits for image processing. With TensorFlow and Keras training a neural network classifier using the Nvidia RTX206 GPU is a walk in the park.
mnist2

Using the default import of the MNIST dataset using tf.keras, which comprises of 60,000 handwritten digits images in 28 x 28 pixels, the training of a neural network to learn classifying it could be accomplished in a matter of seconds, depending on the accuracy. The same learning done on ordinary CPU is not as quick as GPU for architectural differences. In this sample run, the digit “eight” is correctly identified using the neural network.
mnist4.PNG

A simple comparison of the training result of the MNIST database on my RTX2060 with varying training samples depicts slight differences in the final accuracy.
mnist1

 

Monte Carlo methods in TensorFlow

The Markov Chain Monte Carlo (MCMC) is a sampling method to sample from a probability distribution when direct sampling is not feasible.

The implementation of Monte Carlo in the TensorFlow Probability package included sample to run the Hamiltonian MCMC, which is a variation with input from the Hamiltonian dynamics to avoid slow exploration of state space.

While running the samples, my Python reported errors on importing tensorflow_probability:tfp1

The problem is resolved by uninstalling and re-installing the tensorflow-estimator package:


pip uninstall tensorflow_estimator
pip install tensorflow_estimator

Finally the samples run fine with expected results.

tfp3.PNG

The results from the sample run.
tfp7tfp4

Binomial Tree methods for European options using GPU

Binomial methods are versatile in pricing options for it is suitable for American, European, and Asian options. With an European call option with maturity t, strike price k, spot price S, volatility σ, risk-free rate r:

binom1.1

For put option, the last effective term shall be in the max function shall be:
binomput2

Stock’s increment, decrements, and probability to move up are given by the below respectively:

binom2

One of the CUDA samples from Nvidia is to implement the binomial model on GPU.

binom3

 

Test driving Nvidia RTX 2060 with TensorFlow and VS2017

Finally my new laptop that sports the new GeForce Nvidia RTX 2060 arrived. It is time to check out the muscle of this little beast with the toolset I’m familiar with.

On the hardware, the laptop is a i7-8750 and 16G RAM with a Turing architecture based GeForce RTX 2060.

rtx1.PNG

The laptop came with full drivers installed. Nevertheless I downloaded the latest drivers and CUDA for the most up-to-date experience. The software include Nvidia GeForce drivers, Visual Studio Express 2017, CUDA Toolkit, and TensorFlow.

rtx6.PNG

Be careful when trying all these bleeding edge technologies, not only because TensorFlow 2.0 is currently in Alpha, compatibility issues may haunt like with previous 1.x TensorFlow on CUDA 10.1. I have to fallback to 10.0 to have TF happy with it (although one can always choose the compile from source approach).

rtx2

And here are my favorite nbody and Mandelbrot simulation, and also the Black Scholes sample in CUDA. The diagnostic tool in VS gives a nice real time profiling interface with graphs.

rtx5

Finally for this test drive – TensorFlow with GPU. The installation is smooth until I tried to verify TF on GPU. After several failed attempts I realized it could be that CUDA 10.1 may not be compatible with the TF version installed. There are couples of suggested solutions out there, including downgrading to CUDA 9, but since my GPU is the Turing series this is not an option. Actually TF supports CUDA 10 since v.13. So I finally decided to fall back CUDA to 10.0 from 10.1 and it worked!rtx7