Plateau on improving DL and ML skills

Show the Community!
1

Hi everyone,
For the past couple of days I have been stuck in this place in my DL journey where I am not improving. I am kind of stuck. I don’t know what to do. I can implement my own model using TensorFlow but I still need to look at how others implement it first, kind of scared into diving right into coding. I am somewhat okay with math but I would like to improve upon it. I also promised myself that I will be a Kaggle master by the end of this year but I am nowhere near my target. I am scared . Is there anything I can do to get out of this situation where I don’t feel helpless ?

7 Likes
2

Unfortunately, I don’t have a direct answer that can help you with your deep learning journey.

I’ve just come here to say that you’re not alone. I’ve felt exactly like you’re feeling right now so many times in the past, especially when trying to learn something new.

I always turn to this video for motivation when I’m not too happy my progress and feel like I’m stuck in a pleateau, just like you are feeling. Hopefully, it helps you as well. :slight_smile:

https://vimeo.com/24715531

5 Likes
3

Thank you for the video. It was helpful.

4 Likes
4

There’s no harm in reading other peoples code, if anything I think it’s encouraged. One thing that always helps me when I’m plateauing in my learning journey is to remind myself why I started learning ML (because it’s ridiculously cool) and to look back on everything that I have already learned. There’s no stress, keep working at it and keep trying to keep it fun and exciting!

3 Likes
5

As for improving upon math, there’s a playlist on our YouTube channel by @charlesfrye that I recommend: “Math for ML”

6 Likes
6

You are not alone. It’s been more then 3 years and still don’t know where I stand. But the good thing is I like practicing it and I’ll keep practicing forever.

1 Like
7

I’m a practical person, so here are some practical suggestions:

  • don’t be disappointed if you don’t get Master Competition rank this year - getting one gold and two silver medals is difficult, and many people spend years trying to do it;
  • try looking at Pytorch. I don’t imply that it is better, but different people prefer different frameworks. Maybe TenforFlow will be better for you, maybe Pytorch;
  • I don’t know how you study DL, but I suggest trying the fast.ai course. The fastai library is controversial, but the course is excellent;
  • practice writing the whole code for training a simple neural net by yourself. The goal isn’t to blindly memorize the code, but to understand each step: why is it necessary, how it works, and how it interacts with other steps;

Now tips for Kaggle. First of all, I think that you understand what is necessary to succeed on Kaggle. There are a lot of important ideas, of course, but I want to mention three:

  • learn to read the code of other people and learn to do it quickly. You’ll need to go through dozens of kernels through many blogposts with code and repositories to get new ideas. It won’t be easy at first, but it is a skill that can be developed. And be prepared to the fact that there is a lot of bad code;
  • learn to iterate fast over ideas and learn to fail. It is completely normal to try ten ideas and find out that eight of them don’t help. You need to get used to it and not become dispirited when your new cool idea doesn’t work at all. Try the next one, and the next, and the next;
  • competitions aren’t only about generating cool features and training complex models. They are about working with the data, postprocessing, and many other things. You need to be able to do it;

And now more practical ideas about Kaggle.

  1. On developing general Kaggle/DL skills: take a finished competition and try to reproduce some of the top solutions and improve them.
  2. On taking part in an active competition:
  • take several notebooks with the best score and understand how they work. And by this, I mean that you should understand how each line in the code works, what are the ideas behind every decision;
  • try to combine the ideas from these notebooks to improve the score;
  • fix all random seeds to ensure complete reproducibility;
  • read all new notebooks and new comments on the forums daily to find new ideas and inspirations;
  • looks for new ideas in the previous competitions, in the arxiv articles, on paperswithcode, etc.;
  • experiment and don’t give up;
  • have good hardware :smiley:
7 Likes