Research advices and tips for PhD study

Here is a short note about a (subset) of the questions raised in the AIStats'21 mentorship session and my answers. They are about different research advices and tips for PhD study.

Disclaimer: This is far from a complete list, but just a summary of the mentorship session. If I have time I will try to make it more comprehensive later. 

How to pick a good research topic?

Do not pick topics that are super hot. PhD is about to work on one research direction persistently, dig deep and become an expert in that particular direction. Working on hot topic can be quite frustrating since new paper keeps coming up and the performance you currently have may be behind.

At the beginning of the PhD, the most important things is to raise the confidence level. Publishing the first paper means a lot: it gives confidence and enables further collaborations with other people. So there is a trade-off between aiming high and reaching the first milestones. On the other hand, procrastinating things to a few years later would lead to much high pressure (e.g., from the advisor, the family and the peers) and the PhD experience is likely to be not great.

Also, pick topics that are suitable for you. Think about your existing skills and required skills for this research directions. It is great if you could pick a direction that you really enjoy (and constantly get intrinsic rewards). If that's hard, pick direction that are useful and can potentially make good impact.

You can also try to negotiate with your advisor to pick a topic that is of mutual interest (and win-win situation). Take myself as an example, I am personally strong technically but was super bad in presentation and writing due to introverted personality and language barrier. So I ended up with a direction that is different from my advisor, but reached regular coaching for presentation and writing. I spent 2 full months in preparing the first presentation (a CVPR oral talk), memorizing every word (otherwise I won't be able to spit a single word in front of a huge audience). After that things became much better and easier. From my advisor's point of view, the paper can be good milestones for grants so he is also happy with it.

You don't need to make that decision alone. You can ask senior PhDs, other professors, etc. Make connections with other people and listen to their advice and thoughts.

How to learn new directions with no prior experience?

Read papers, read surveys, keep reading them. Don't memorize them, think about the underlying logic chains and how the story flows.

One tip is to "guess" or "predict" what the paper will say next, without passively reading them. This will force you to think about the logic (like self-supervised learning) and at the same time, accelerate paper reading later substantially. This will also naturally lead to novel research ideas/thoughts once you have read enough papers and summarized them well.

Another tip is to write 1-2 sentence summary of the papers read (or tell other people the content of the papers). Once you realize you cannot summarize well, it is time to go back and re-read the paper. Reading alone doesn't count since the understanding may not be there.

At the beginning, the summary can quite low-quality, overly focusing on small details etc. But things will be substantially better if keeping this habit for long enough.

You can also ask other people who are experts in the field. Think about what you already know, and ask questions proactively. For example, ask about the top3 works in the field and why they are impactful, etc.

Note that the meta-skill of "ramping up a new direction quickly" can be super important for the rest of your life. The research direction you worked on in your PhD can be obsolete very soon, but the learned meta-skill enables you to pick up new things very quickly and make you be adapted to new situations. This is one of the big things I learned from PhD life.

I personally switched from physics-based vision (as my PhD topic) to deep reinforcement learning after I joined FAIR. It was a very nice experience and a lot of things need to be done in a proactive and self-motivating manner. Hands-on capability (e.g., strong coding) definitely helps when there were no PyTorch/Tensorflow and no existing codebase to reference from. Math skills is also very important to nail down the gist of the idea and reason about things clearly.

Always remember that freedom is not granted but earned, if acting fast enough.

How to make full use of the focus time? What becomes different once you enter industry?

How to make efficient use of the focus time is another important meta-skill to be learned in PhD. Note that it is very hard to do that at the beginning. People who are motivated to do a PhD are often lured by the big, open problems in the field and thinks they can solve them in a few years. Such a high aim may sometimes create negative / detrimental effects on their PhD life.

I personally experienced a lot of "wasted" thinking at the beginning of my PhDs. E.g., I spent 40 hours thinking about useless stuff and another 40 hours doing actual works that are immediate or asked by my advisor. This is a very frustrating experience.

Fortunately, things will get better over time. Here are some tips. You can break a demanding goal into small steps until they are reachable. Lists possible milestones and arrange them properly. Formulate (or nail down) what you have thought about clearly in a piece of paper. Give time to take a rest and refresh, etc. To be honest, many interesting thoughts hit me in the shower time.

Sometimes reading literature will also trigger new thinking and thoughts. I have experienced these many times, in particular when what I predicted for the rest of the paper is different from what actually is written in the paper. Such a "what if" thinking will naturally lead to nice insights, and is quite efficiently by itself.

The reason why such an "efficient focused thinking" is important is that after PhD graduation, you won't have the luxury to spend 40 hours thinking about useless things anymore. You would need to take care of the family, to deal with a lot of regular errands, to be interrupted constantly by emails, calls, messages, etc. In this case, if you have already learned how to think efficiently, it will bring tremendous advantage to your future career.