How to do research

Choosing the right research problem + how to succeed 🎓

How to do research

Choosing the right research problem + how to succeed 🎓

Choosing the right research problem is difficult. Both Vladlen Koltun and Richard Hamming offer some great advice on selecting which research problems to work on and how. This post is simply a summary of Richard’s talk “You and Your Research”, and Vladlen’s talk “Doing (Good) Research”, both worth watching in their entirety!

Problem

Progressing from undergrad into research can seem to flip any STEM field from a science into an art: undergrads identify solutions to problems versus researchers who identify problems worth solving. The right problem is ill-defined, yet critical to identify.

Define “right”

Richard: right is simply subjective, up to you.

Vladlen: right should:

  • Contribute to the progress and well-being of humanity, should be useful.
  • Maximize your contribution to the research community and society (not vice versa) by
    • inspiring the community;
    • shifting the way the community thinks about existing problems (or new ones);
    • be a methodical evaluation of methods to demonstrate which should be used when;
    • be a dataset, benchmark, or open source code people find useful.

How to work on the “right” problems

  • Create a long term vision to guide yourself (not a random walk), have high-level meta goals.
    • Ask yourself: what fields or applications do you find interesting or meaningful and why?
    • You’ll only really succeed in work that you enjoy.
    • Doing meaningful work helps avoid burnout.
  • Understand what components are needed for your high-level goals to become a reality.
  • Understand the state-of-the-art for each component by reading and doing (explained later).
    • Analyze bottlenecks.
    • Understand what you could do to benefit the collective progress of the community.
  • Solve a component’s problem if the time is right.
    • Too early = not sufficient tools to attack the problem well yet, will make little headway.
    • Too late = you won’t contribute to affecting the research community’s direction much, improvements will be too incremental.

Understand by reading

  • Read a lot.
  • Read critically. When reading about a new method, ask:
    • What are its limitations and assumptions, when will it break?
    • What gaps remain?
  • Reading critically enables new ways of thinking about methods. Otherwise you won’t make big changes, you’ll merely extend the old with incremental improvements.
  • So tolerate some subjective uncertainty about the “accepted methods”. Be sufficiently skeptical to perceive flaws and see what others missed. It’s OK to be (justifiably) contrarian!
  • Inform your colleagues about your research interests. They’ll give you greater observability of the relevant papers you should read (but missed).

Understand by doing

  • Re-implementation is the best way to understand a method. You’ll discover details you can’t glean from just reading, and develop deeper insight.
  • Implementation helps observe things you never set out to discover. And by following the scientific method down this rabbit hole, you can then go about testing assumptions about what generated the effect you observed.
  • Research then begets research: the more things you try, the more you observe, the more things you want to try next etc, the more people will want to collaborate with you, and invite you to things, and tell you about things you didn’t otherwise know.
  • Be methodical when (re-)implementing: swap a method’s subcomponents in and out in a controlled way to understand which components were critical.

Understand by writing

  • Write down your ideas + experimental methods early.
  • This makes clear what your assumptions are and what gaps remain to be filled.
  • You can also share this (more concrete) document of your ideas with your peers.

jpg

Work ethic

  • Luck favors the prepared!
  • Research is 99% perspiration, 1% inspiration. Newton thought if people just worked as hard as he did, they’d get the same result.
  • Great researchers always think about problems, even when they’re away from work.
  • Set aside quiet time reflection time for “great thoughts” like Friday afternoon to:
    • Think higher-level about where you are heading, is this the right direction?
    • Don’t cling to bad ideas too long, know when to walk away / bury work.
  • Have 10–20 problems in your head at any one time, so you don’t stay “stuck” in one problem. When a clue comes along for one of them, focus on it.
  • Have an open door policy (need to collaborate), it’s better in the long run.
  • Learn how to communicate your ideas in a formal and casual way.
  • Be on the lookout for collaborators.
  • Quality over quantity (do not add noise to conferences or arXiv, or worse: mislead).
    • Contribution is the goal, the publication process is a liability.
  • Confidence is necessary: it is important that you believe you can do great work.

Challenging yourself

  • Ask yourself: if what I’m doing is not important, and not likely to lead to important things in the future, why am I working on it?
    • Caveat: not to say adopt “Nobel prize winner syndrome” of only working on “great problems” and getting nowhere.
    • Instead, work on small acorns that have the potential to grow into mighty oaks.
  • Ask yourself: what are the most important problems in my field?
  • Be willing to accept change, don’t stick to a particular method.

Warnings

  • Sometimes the people around you cannot see you’re doing great work.

A printer-friendly version of this post is here.

jpg

Avatar
Rowan McAllister
Machine Learning Manager

My research interests include autonomous vehicles, reinforcement learning, and probabilistic modelling.