Real time One Time Password recognition with Nvidia Jetson Nano

The Jetson Nano is a machine learning development board from Nvidia. It sports a 128 core GPU along with ARM CPU running Ubuntu. It is great for neural network based AI development and prototyping intelligent IoT applications.

For the 2G model, it is powered by a Type C USB port. Power supply unit is not included. Meeting the minimum of 5.1V at 3A power requirement will guarantee best performance, while ordinary power supply for phones should be enough for average usage.

An interesting application with the pre-installed OpenCV is to read One Time Password (OTP) from phone. OTP is part of a security design known as multi-factor authentication (MFA). It has been long used by applications requiring higher level of authentication requirements like online banking. Before smart phones, OTP devices known as hard token are distributed to end users. Nowadays, applications installed in smart phones is the mainstream. The idea of OTP is simple, it is a series of digits, usually comprised of six, that will change every minute. End users are to provide this OTP in addition to user name and password.

With the Jetson Nano and OpenCV, the only additional software to install is Tesseract from Google. It is an OCR application that can run on multiple platforms, including Ubuntu. Although it is not optimized for CUDA, it still perform satisfactorily as shown in the below test with Fortinet application on Android phone.

The camera used is a LogiTech c270 camera which is officially supported on the Jetson platform. The focus is not adjustable for it is designed as a general purpose web cam. Even with a blurred video feed, recognition rate for digit is still good.

OpenVINO on Raspberry

OpenVINO is the short term for Open Visual Inference and Neural network Optimization toolkit. There is a port to the Raspberry platform running Rasbian OS.

To setup on a Raspberry Pi, download the latest zip from OpenVINO, and run the commands below.

sudo tar -xf l_openvino_toolkit_runtime_raspbian_p_2019.2.242.tgz --strip 1 -C /opt/intel/openvino
sudo apt install cmake
source /opt/intel/openvino/bin/setupvars.sh
echo "source /opt/intel/openvino/bin/setupvars.sh" >> ~/.bashrc
sudo usermod -a -G users "$(whoami)"
sh /opt/intel/openvino/install_dependencies/install_NCS_udev_rules.sh
mkdir build && cd build
cmake -DCMAKE_BUILD_TYPE=Release -DCMAKE_CXX_FLAGS="-march=armv7-a" /opt/intel/openvino/deployment_tools/inference_engine/samples
make -j2 object_detection_sample_ssd

 

Once the installation completed, download the pre-built model for facial recognition. The following test will return an output image with the face detected using an input image.

wget --no-check-certificate https://download.01.org/opencv/2019/open_model_zoo/R1/models_bin/face-detection-adas-0001/FP16/face-detection-adas-0001.bin
wget --no-check-certificate https://download.01.org/opencv/2019/open_model_zoo/R1/models_bin/face-detection-adas-0001/FP16/face-detection-adas-0001.xml

./armv7l/Release/object_detection_sample_ssd -m face-detection-adas-0001.xml -d MYRIAD -i barack-obama-12782369-1-402.jpg 

 

 

Visualizing a MLP Neural Network with TensorBoard

The Multi-Layer Perceptron model is supported in Keras as a form of Sequential model container as MLP in its predefined layer type. For visualization of the training results, TensorBoard is handy with only a few line of code to add to the Python program.

log_dir="logs/fit/" + datetime.datetime.now().strftime("%Y%m%d-%H%M%S")
tensorboard_callback = tf.keras.callbacks.TensorBoard(log_dir=log_dir, histogram_freq=1)

Finally add callbacks to the corresponding fitting model command to collect model information.

history = model.fit(X_train, Y_train, validation_split=0.2,
epochs=100, batch_size=10
,callbacks=[tensorboard_callback])

Once the training is completed, start the TensorBoard and point browser to the designated port number.

Click on the Graph tab to see a detailed visualization of the model.

Click on the Distributions tab to check the layer output.

Click on the Histograms tab for a 3D visualization of the dense layers.

 

 

Experiencing Deep Learning with Jupyter and Anaconda

Most of the time my work with deep learning is done in command line interface with Python and TensorFlow. The clean and efficient syntax of the Python language and package design of TensorFlow almost eliminated the need of a complex Integrated Development Environment (IDE). But after trying out the free Google Colab service that provide a web based interface in Jupyter, I am going to set up one on my desktop that sports an Nvidia RTX2060 GPU.

Installation is easy, but be sure to run Anaconda console as Administrator on Windows platform. For running TensorFlow with GPU:

conda create -n tensorflow_gpuenv tensorflow-gpu
conda activate tensorflow_gpuenv

Managing multiple packages is much easier with Anaconda as it separate configurations into environments that can be customized. On my development machine, I can simply create a TensorFlow environment with GPU and then install Jupyter to enjoy its graphical interface.

Finally to activate Jupyter:

jupyter notebook

To see how Anaconda with Jupyter is flexible on the same machine, a comparison of a simple image pattern recognition program runs under Jupyter with and without GPU support.

Machine learning with a classic game Doom and TensorFlow

Busy this weekend with work. During those irregular shift, I decided to let my GeForce RTX 2060 to play some game while I am away from home.

VizDoom is a port of the classic first person shooting game Doom to the machine learning area. With this specialised port compatible with Python, it is easy to train neural network to play this human game with TensorFlow. Over the weekend, my GPU card played countless episodes of games to learn how to win on itself while I am working in office.

 

Visualizing TensorFlow with TensorBoard

TensorBoard is a tool for visualizing graphs and various metrics for TensorFlow session runs. With a few line of additional code, TensorBoard gathers and report run statistics in a nice graphical interface.

First of all a few lines are required to TensorFlow session as in below:

writer = tf.summary.FileWriter("output", sess.graph)
print(sess.run(GH, options=options, run_metadata=run_metadata))
writer.close()

Before running the session, start TensorBoard using the following command.

Finally run the TensorFlow session and point the browser to the TensorBoard.

Profiling machine learning applications in TensorFlow

TensorFlow provided package timeline by using the import from tensorflow.python.client

from tensorflow.python.client import timeline

This is useful for performance profiling TensorFlow application with graphical visualization similar to the graphs generated from the CUDA Visual Profiler. With a little tweak in the machine learning code, TensorFlow applications can store and report performance metrics of the learning process.

The design of the timeline package made it easy to add profiling by simply adding code below.

run_options = tf.RunOptions(trace_level=tf.RunOptions.FULL_TRACE)
run_metadata = tf.RunMetadata() 

It is also required to instruct the model to compile with the profiling options:

model.compile(optimizer='adam',
loss='sparse_categorical_crossentropy',
metrics=['accuracy'],
options=run_options,
run_metadata=run_metadata)

With the sample mnist digits classifier for TensorFlow, the output shown Keras history are saved and can be later retrieved to generate reports.

Finally, using the Chrome tracing page ( chrome://tracing/ ), the performance metrics persisted on file system can be opened for verification.

 

More test driving of Tensorflow with Nvidia RTX 2060

By following the TensorFlow guide, it is easy to see how TensorFlow harnesses the power of my new Nvidia RTX 2060.

The first one is image recognition. Similar to the technology used in a previous installment on neural network training with traffic images from CCTV captured, a sample data set of images with classification of fashion objects are learnt by using TensorFlow. In that previous installment, Amazon Web Service cloud with a K520 GPU instance is used for the model training. In this blog post, the training is actually taking place in the Nvidia RTX 2060.

Another classic sample is the regression analysis, from the Auto MPG data set. With a few line of code, TensorFlow clean up the data set to remove unsuitable values and convert categorical values to numeric ones for the model.

Interesting applications of Generative Adversarial Network to crack CAPTCHA

Interesting results from a recent paper presented at the 25th ACM conference on Computer and Communications Security shown advances in Generative Adversarial Network (GAN). In particular the paper focused on tackling Captcha with GAN. GANs take a game theory approach in the training of network and during the deep learning process two entities compete in a game that one trying to fool the other while the other strives not to be fooled.

Comparison of performance of machine learning the probability distributions are usually considered as metrics for benchmark. One such commonly used is the Jensen–Shannon Divergence and a generalization can be given as

 

Movidius compute stick on Raspberry Pi

The Movidius compute stick is supported on the Raspberry Pi 3 Model B platform. This is almost like a dream deployment for low power consumption applications. The overall dimension of Movidius with Raspi in their original configuration might not fit into some commonly available IP56 waterproof enclosures but with some ultra short extension cables and converters, packing them insides and even with a lithium battery pack should be easy.