PYTORCH VS TENSORFLOW: COMPARISON BY APPLICATION AND FEATURES

Since the day deep learning has taken off, there have been advancements in the development of new frameworks at a blazing speed. Consider the following timeline.

Image credits: http://www.popit.kr/wp-content/uploads/2017/05/deeplearning_fw_timeline1.png

In 2010, Theano was released and was used widely as a framework for deep learning tasks. It was not until three years later; large companies started taking an interest in the development of deep learning frameworks. Berkley AI Research and other contributors developed Caffe. Torch was initially released in 2002 and was written in Lua and C++.

Tensorflow was developed as one of Google's internal use in the year 2015 by Google Brain. It was later released as an open source library. While Pytorch was released as early as October 2018 by the Facebook team. So, coming to the point - Which one is for you - Pytorch or Tensorflow? To answer this question, let's look at how these two frameworks differ.

Kind of programming

The most popular reason for using Pytorch is its ease during prototyping and experimentation. Pytorch code is imperative. Meaning, as you write code, it is being executed step by step. So what's different about tensorflow? Tensorflow is declarative. This means that when you define a set of operations in lines of code, the code is not executed on the go. First, you need to compile your code, and then all of the execution happens in a single go. There are a few advantages of this type of programming in terms of efficient memory usage, over the imperative approach of Pytorch. But like mentioned before for fast prototyping, Pytorch is a clear winner.

However, the recent release of Tensorflow 2.0 has announced that Tensorflow has now adopted Keras as its high-level API. Tensorflow 1.0 is a low-level API. Meaning, you have to write a lot of code from ground level. Keras was built on top of Tensorflow earlier to ensure that standard implementation of Neural Networks did not require much code. Keras API wrapped tensorflow code to make the implementation developer friendly. However, Tensorflow has now adopted Keras as its official high-level API which is available as tf.keras. If you are a beginner, tf.keras should be your first choice to learn a lot of practical deep learning quickly. Besides, tf.keras moves away from traditional Tensorflow declarative style of programming. Meaning, if you add two numbers in a line, the numbers get added. It does not wait for the graph compilation like in the 1.0 version.

Prototyping and debugging

Tensorboard, a beautiful GUI to visualize different aspects of your neural network like losses, weights, and gradients are widely used to improvise model's architecture. It is a blessing from Tensorflow and makes it an excellent tool to use. In the case of Pytorch, there is no such inbuilt visualization tool in its native form. But there are popular third-party libraries available to enable tensorboard in Pytorch.

Image Credits: https://www.tensorflow.org/images/mnist_tensorboard.png

Production Environment

We talked about how the two frameworks work in a testing and research environment. However, what about the production environment where you need to serve models at large scale?

Tensorflow has certainly tested frameworks like Tensorflow Serving and Tensorflow Lite (for mobile applications). Tensorflow serving helps developers serve a model in a production environment after it has been prototyped and tested. TensorFlow Serving makes it easy to deploy new algorithms and experiments, while keeping the same server architecture and APIs.

So what about Pytorch? Well, Pytorch has a feature called ONNX (Open Neural Network Exchange) which enables using Pytorch model in a production environment. It can export Pytorch models to Caffe framework, which is made for mobile and massive scale deployment.

New learner’s angle

If you are completely new to deep learning and are looking to build something really quick, use keras. Keras code is very readable and easy on the eyes. If you wish to do research or implement novel models by yourselves, by all means, use Pytorch. It enables you to write Pythonic code. This makes it easy to learn. If you want to make models integrate with your app or product, go with Tensorflow. The community is stronger than that of any other framework. There is a lot of help available online in the form of GitHub issues or tutorials for almost any Deep Learning tasks. Plus, it offers a suite of tools to make your model production ready. So once you are familiar with the implementation using Keras, you can switch to low-level tensorflow APIs.

Conclusion

Hope you have a basic idea of Deep Learning frameworks. It can be a little confusing in the beginning. However, here's what you need - A lot of practice and guidance. Without a balance of these two, it is not possible to be industry ready. We at Manipal ProLearn make sure that you get the right mix of these two. Check out our Data Science and AI courses at: https://www.manipalprolearn.com/data-science/