Hi folks, super excited as we are going strong with our AMA series!
The guest
Our second AMA will be with the amazing Yannic Kilcher.
Yannic Kilcher is an ML & AI content creator and one of the biggest YouTuber in the ML space with his lightning speed but detailed paper reviews. He holds a PhD in Artificial Intelligence and loves to study the theoretical and practical behaviors of learning systems and optimization algorithms, as well as their interactions while working with both large as well as high-dimensional data.
Visit his Youtube channel.
He is the CTO at DeepJudge and is also co-host of ML Street Talk where they interview influential ML researchers.
Heās looking forward to answering all your questions about content creation, paper reviews, the fascinating world of AI, and more.
How to participate
Post your questions as replies to this thread ā you can post anytime between now and the start of the AMA.
At 11am on Nov 18, Yannic will pop into the community forum and answer your questions for an hour!
What do you think is the most underrated area of ML research at the moment? What advice would you give to people just getting started doing ML research?
thanks for taking the time to answer our questions.
when you started your Youtube channel, did you ever imagine to grow so big and become as influential?
And how do you find a balance between your work as a CTO, content creation, research and free time?
Hi Yannic ,
Nice to meet you.
I feel that in many sciences there is a gap between the theories developed in academia and the practices used that often come from experience learnt by solving real business problems. Since you spent many years in academia and recently moved to the business world, do you believe there is similar a gap in machine learning?
From this idea, do you think that having a deep understanding of the theoretical concepts and formulas of a ML technique is always necessary to use that method to solve concrete business problems or is it one those things that itās nice to have, but itās not necessary?
Thanks Yannic for answering my questions.
Keep it up the good work on Youtube (Iām huge fan of your contents ) and I wish you all the best for your new role at DeepJudge.
First, I am a big fan of you YouTube channel. Thank you for producing such great content for the ML/AI research community!
I am curious about what you think of the future of reinforcement learning (e.g. in 10 years)? Actually, I am currently a PhD student working on RL problems. I can see that there is great potential in it, but it seems there is also much difficulty in making it work in many real world applications. What do you think of current situations? How do you think we may get over the challenges in the near future?
Sorry for being a bit talkative, but I actually have one more question regarding the career development in ML research.
How do you see current job market in industry for ML/AI researchers? Are there any important suggestions you may have for students like me who want to work as an ML researcher in industry after graduation?
Looking forward to knowing your idea on this. Thank you!
I am a huge fan of your videos and I always wonder how you can be so fast when creating them and that is why I would love to know more about this process. How long do you spend (on average) reading a paper before starting your review? What leads you to choose a specific paper for your next video? (knowing that your videos can influence the impact of a paper). Do you also share the feeling of not getting the full understanding of a work until you fully code it or are you definitely above this level?
First of all, I wanted to show appreciation for your videos, they have helped me a lot. One thing Iāve noticed while following you is the diversity of the papers you have explained. For some that could sound like a joke given that most of them are from the area of ML and specifically DL but knowing how hard it is to state up-to-date with academia even in a small niche I wonder, how do you get updated or discover new research and how do you filter research to read out of your main area of expertise?
Anything you want to add to your research workflow (apps, devices, resources, anything that could be useful) will be appreciated.
First of all, thanks for doing another AMA on our forums! Itās so awesome!
As a fellow content creator, Iām curious, do you find any inspiration from the broader creator community? Itās really hard to be creative for me personally when presenting technical content.
Do you have a planning process that goes behind the scenes? Do you follow a methodology or are the videos more impromptu?
Hi
Underrated? Very hard to say, as Iām probably subject to the same biases as everyone else. I personally really enjoy work in multi-agent RL, world models, and planning, but I have no idea if that will ever work well.
For people just starting out: Try to ask the fundamental questions. Donāt give into every latest hype, and try to understand a few things deeply, rather than understanding most things shallow-ish.
Hey!
I never even imagined more than 20 people subscribing. I did realize that there was a lot of content for beginners on youtube, and also a lot of content for researchers (conference proceedings), but there was nothing bridging the gap outside of a university education. But I wouldnāt have thought that there was an actual demand for filling that gap. I mainly started because it forced me to clarify my thoughts, and I had to read some obscure papers on RL, so I thought maybe it will help someone else who has to read the same papers.
I generally am terrible at time management. I usually respond to stuff 3 months late, I forget lots of things, etc. I donāt have it figured out quite yet. Iām a CTO during the day, then some sports and some cooking and every free minute after that goes towards the channel in some way. Some days thatās zero, but other days I manage to find some time. But if anyone has actually figured this out, Iād love to know
Hi Matt
This gap definitely also exists in ML. Even within research, there is a huge gap between the theoretical papers, which usually have to make ridiculous assumptions, and the practical papers. Even those practical papers then are mostly irrelevant for business because most donāt work. They find some .1% improvement on some benchmark, but itās not robust at all, or flat-out fake. ML has a redeeming quality though, since there are a lot of industry-players in research (Google, Meta, etc.) and that usually leads to results that can be applied in industry, so I guess in total itās somewhere in the middle in terms of how bad the situation is.
As for having a deep understanding, I generally am pro understanding things because I think it leads to better diagnoses of problems and more creative solutions, but I see many many people in industry just hacking around bit by bit and solving most of the problems to a sufficient degree, maybe even better because theyāre usually more pragmatic, so I wouldnāt say itās a necessity.
Hi
RL is a mixed bag because if you look at it from first principles, itās hard to see how it could ever work in its original formulation (environment āā agent) because you quickly run into some no free lunch problem. I think what a lot of RL is missing is either the introduction of strong priors, or efficient methods to acquire such priors, such as a core understanding of physics, or time, or the inherent notion of objects, etc. I think people underestimate just how important those things are when they consider, for example, how humans learn new tasks. But in general I love RL and I think weāll make great progress because it can be applied in so many places.
Yea the ML researcher question is a good one. There used to be a time (ca 2018) where the big companies would hire every single decent phd graduate as a researcher into their labs, but that time is more or less over. If you really want to continue doing research, you either have to be exceptional, or youāll have to start working as an ML engineer / software engineer and try to get into a team that also publishes. Research publications arenāt that interesting to companies, itās a bit of clout and they can attract good people by promising that they can do some research work, but beyond that, itās not really attractive for them.
Hi
I usually read a paper until I understand all of it. Sometimes thatās twice, and sometimes thatās six times. The āhow do you chooseā question comes up often, but itās really just my interest. I donāt care that it might influence anything. Other people can voice their opinions just as well, or evaluate for themselves if I make good arguments or not.
In general, I donāt code up the papers, but I would definitely profit from it and gain an even deeper understanding. I donāt feel that level is necessary for what I do, but yea maybe I should do it sometimes.
Hi
I struggle with keeping up to date like everyone else. I have many streams, arxiv, twitter, discord, reddit, blogs, newsletters, etc. and Iām obviously subjective and biased, so I donāt claim I have a well-calibrated filter. So itās just my interest that decides, and a lot of randomness. I also try to expand my area of expertise, rather than trying to reduce the papers I look at. Itās more fun that way
My workflow is non-existent, I download papers, I use Pocked, but beyond that, nothing, no organization, no note-system, nothing.
I definitely find a lot of inspiration from other creators. From the ML creators I see other approaches than mine to explaining things, and often it gives me new ideas on how to improve my explanations, or I learn something myself about a new research direction. From the more broader creator community, I get inspiration on how to expand my style. Iām a big fan of all kinds of youtubers, casey neistat, pewdiepie, dream, veritasium, and I generally enjoy thinking about how I could adopt a bit of their style into a video.
My videos are generally all impromptu, I read papers and then I record. For the news I collect during the week, then at the end I go through it all, order it, filter it, and then just speek freely what my opinion is.
Also, as a general statement to everyone who has thanked me for the content, or wished me well, I wanted to get through all the initial questions quickly, but thank you all so much, it is very kind of you and very good to hear that the content is helpful and / or entertaining. Thatās ultimately what counts and nice to see that thatās actually the case.
I guess time is slowly coming to an end if anyone has more questions, feel absolutely free to ping me on twitter or discord. I also generally try to read all youtube comments, but I donāt have time to respond to each one, and sometimes I do miss some (for which Iām very sorry)
,
) and I wish you all the best for your new role at DeepJudge.
