by wpmoradi on 10/8/18, 4:28 PM with 28 comments
by krat0sprakhar on 10/8/18, 6:09 PM
I took a class under Bjarne Stroustrup and he highly recommended Tour of C++ [0] as the best way to learn modern C++ for someone who already has some programming experience. That and of course, Effective C++[1] by Scott Meyers.
[0] - https://www.amazon.com/Tour-C-Depth/dp/0321958314
[1] - https://www.amazon.com/Effective-Specific-Improve-Programs-D...
by siekmanj on 10/8/18, 5:13 PM
That said, I think one of the best ways to learn how to do something is dive right into it. I'm currently in the process of writing a deep learning library in C: https://github.com/siekmanj/sieknet
So if you're dead set on C++, I'd say the best way is to try to build something cool right off the bat.
by spott on 10/8/18, 8:23 PM
This will kind of define how to go about doing it.
I can think of a few different reasons you might want to do this:
* You DL code in another framework is slow, and you have some custom C++ you want to write to speed it up. For this, you probably want to learn both high performance C++ and/or CUDA kernel development. You can probably avoid diving completely into C++, and just call a couple of optimized routines from python (and whatever other deep learning framework you are using in python). In this case, it might be worth looking into Tensor Comprehensions.
* You are taking someone elses DL algorithm and moving it to C++. In this case, you still don't need to be diving too much into C++, Ideally, you probably want to use something like ONNX and Caffe, and then call into them with some python wrapper. If you want the whole thing to be in C++, then you are still going to use something like ONNX and Caffe, but you will be writing mostly application code to support it.
* You want to help with the development of Pytorch, Tensorflow, Caffe or anther deep learning framework. In this case, you need to know CUDA kernel development, performant C++ idioms, and good C++ application development idioms.
These are pretty different flavors of C++, which makes it hard to give you a good answer.
by valedra on 10/8/18, 5:51 PM
by make3 on 10/8/18, 7:05 PM
PyTorch was apparently used in ~43% of the ICLR papers (a big prestigious deep learning conference) (https://www.reddit.com/r/MachineLearning/comments/9kys38/r_f...) and it's market share is growing at an incredible pace. I would expect it to beat Tensorflow in popularity at least for research very soon.
That's definitely where I would start if I absolutely had to do deep learning in C++. Otherwise I would do Python, which is likely used by 95% + of the research field.
by madenine on 10/8/18, 6:16 PM
1. Understanding of Deep Learning
2. General familiarity with the language
3. Understanding of packages/tools that will help you accomplish your goals. If this were python we'd be talking Numpy/TensorFlow/PyTorch/Keras/etc.
We're talking C++, I'm guessing you'll want to work with less abstraction than TF and the like, so BLAS, LAPACK, and maybe the whole CUDA family (CUDA, cuBLAS, cuDNN, etc).
Otherwise, TensorFlow, Caffe, CNTK, and other popular deep learning frameworks are available for C++, even though they might be better known and more heavily used in other languages.
by gmiller123456 on 10/9/18, 5:47 PM
I think it's also important to point out that it's not a great way to ask for help without giving some details about your background and your motives. If you've been programming in any language for quite a while, you'll need a completely different approach to "learning C++ for deep learning" than someone who's never programmed before. Likewise, if your goal is to work in the industry producing cutting edge work, you'll take a completely different approach than someone who just wants a rough understanding of the details just to satisfy your curiosity. So, without providing that context, you're going to get answers that are all over the place, almost all of which will be useless to you.
by mooneater on 10/8/18, 5:53 PM
1) learn c++ 2) learn deep learning 3) learn c++ with deep learning: much easier once you have done 1) and 2)
by ergodic on 10/9/18, 7:30 AM
It is very complete. Similar to pytorch (which borrowed some ideas from Dynet) and backed up by top academia (not big tech, hence its lower profile). It has a python wrapper as well, as everything nowadays.
If you are into deep learning for sequence to sequence (machine translation) Marian is also C++ centered. Fastest MT around. Complete (transformer and the like), extendable. Not that reading friendly from what I have heard.
Disclaimer: Python type of guy here.
by gaius on 10/8/18, 8:55 PM
Also Nvidia have a course https://courses.nvidia.com/courses/course-v1:DLI+C-AC-01+V1/...
But I am also curious why... this is deep-end stuff if you’ll forgive the pun. Why not start with Keras and only dip into the C++ if you need to?
by jahewson on 10/9/18, 6:25 AM
Make sure you learn how to use a debugger. You’ll need it.
by skinner_ on 10/8/18, 7:30 PM
by jason_slack on 10/8/18, 7:27 PM
Stroustrup and Meyers are the two most well known names. I buy all their books and I read them and practice all exercises and problems if the book has them.
by mothsonasloth on 10/8/18, 9:21 PM
Use Java and enjoy C++ like syntax with 80% equivalent performance (based on my experience)
Then try this library - https://deeplearning4j.org/
by wpmoradi on 10/8/18, 4:37 PM
by timwaagh on 10/8/18, 8:47 PM