By Dr. Viral B. Shah Co-Founder & CEO of Julia Computing
Deep neural networks have been around since the 1940s, but have only recently been deployed in research and analytics because of strides and improvements in computer technology and computational horsepower afforded by modern GPUs. Neural networks can now carry out tasks that were previously considered difficult for computers such as vision and speech processing. These are being combined into really interesting applications such as self driving cars, Amazon Echo, personal assistants, bots, and many others. Interesting and exciting as these applications are, businesses today are faced with a number of challenges:
1. A major challenge in the new age of AI applications is safety and regulation. How do we ensure that the algorithms do what they are intended to do? How do we eliminate model risk from financial markets? How do we keep the self driving cars and our UAVs safe?
2. While companies such as Google, Facebook, and Amazon can devote dozens of engineers to fine tune their deep learning models and extract the best performance from their programs, it is simply not easy for the rest of the world to do so.
3. These applications need to run on a wide variety of hardware such as clusters of GPUs for training petabytes of data. The trained models then may be deployed on the web or on smartphones or the tiniest of ARM systems.
There are a number of ways to address these challenges. One popular approach has been to build sophisticated libraries for deep learning. Examples of this approach include MXNet, Caffe, Torch, Theano, and TensorFlow. Given the need for speed, these libraries are implemented in C++ and have wrappers for Python, R and other easy to use programming languages. This two language approach is limiting because it reduces flexibility, and makes it difficult to customize models to the problem at hand. There are hundreds of domains in health, education, transportation, retail, manufacturing and so on, where a one size fits all approach is unlikely
Julia is a language that was designed by its authors - Jeff Bezanson, Alan Edelman, Stefan Karpinski and Viral Shah - to solve exactly this two language problem for data science and machine learning. Choice of language is key. While one may argue that any task can be carried out in any language, an efficient language saves compute costs, and an easy to use language saves human effort.
The Berkeley Autonomous Race Car (BARC) uses algorithms powered by Julia
Julia’s deep mathematical roots make it easy to translate algorithms from research papers into code, with no loss in translation - thus improving safety and reducing model risk. Being JIT compiled, Julia programs run at the speed of C, while still being as easy to program in as Python or R. The most important aspect however is the community. Julia is open source, licensed under the liberal MIT license, and includes contributions from hundreds of programmers and researchers around the world. Julia also runs on all kinds of silicon out there - chips from Intel, IBM, NVIDIA, ARM, and many others that are yet to come.
Leveraging these capabilities in Julia has led to the creation of a number of deep learning libraries in Julia such as Mocha, Merlin, and KNet, while still retaining the ability to use MXNet and TensorFlow. As AI goes beyond research labs and academia, the industry needs tools that are open, easy, well tested, scalable and reliable.
We discuss some very interesting case studies that we have carried out for deep learning with Julia:
Automated Detection of Diabetic Retinopathy
An interesting use case for artificial intelligence could be the ability to drive down costs in medical diagnosis.
Diabetic retinopathy is an eye disease brought on by diabetes. There are over 126.2 million people in the world (as of 2010) with diabetic retinopathy, and this is expected to rise to over 191.2 million by 2030. According to the WHO in 2006, it accounted for 5 of world blindness. Hence, early and automatic detection of diabetic retinopathy would be desirable.
Engineers at Julia Computing took up this problem and built models that would process images of real clinical data. Julia speeds up the entire pipeline from preprocessing, training, post processing and prediction, and leverages heterogeneous compute platforms to provide a fast,reliable solution.
Have you ever wondered how the app Prisma manages to turn your photos into impressionist paintings? They use, amongst other things, a special type of algorithm called neural styles.
Neural styles are a special type of algorithm that combines the content of one image with the style of another using deep neural networks. Here they demonstrate how one can use a class of deep neural networks to extract features from any “style” image and subsequently “apply” them onto any “content” image.
We find that several of Julia’s language features lend themselves naturally to the domain of artificial intelligence. Since Deep Learning is fundamentally changing our lives through interesting products, using a powerful, high performance, open source and easy to use language such as Julia will give those using it a strategic advantage in their respective field.