NEW BOOK!
Explore a better way to work – one that promises more calm, clarity, and creativity.

Yoshua Bengio’s Deep Thoughts on Deep Thinking

Thinking about Thinking

One of the things that surprised me when researching Deep Work was how rare it was to find examples of smart people talking coherently about the process of effective thinking.

This is why I was pleased when a reader recently pointed me toward a video interview with computer scientist Yoshua Bengio — one of the big names in machine learning.

Around an hour and ten minutes into the interview, Bengio turns his attention to the topic of productive thought:

“There is also something to be said about concentration…to really make big progress in science you also need times when I can be very focused and where the ideas about a problem, and different points of view, and all the elements sort of fill my mind. I’m completely filled with this…that’s when I can be really productive.”

Filling your mind with a single problem, Bengio emphasizes, is not a fast process, noting:

“It might take a really long time before you reach that state…”

Hard problems are hard to understand. The best researchers in my field of theoretical computer science, for example, tend to be those who have the drive, brain power and flexibility to read other peoples’ papers constantly to spark a new recombination or extension that advances the field — an exhausting process.

But as Bengio emphasizes, this effort is worth it:

“[After you fill your mind with a problem is] when you can really start seeing through things and getting things to stand together and solving…now you can extend science, now when things are solid in your mind you can move forward.”

Our culture likes to emphasize eureka moments, but we often miss the hard, patient work that goes into preparing the mind to generate breakthroughs.

(Hat tip: Daniel)

26 thoughts on “Yoshua Bengio’s Deep Thoughts on Deep Thinking”

  1. You might also find interesting the book by
    Josh Waitzkin
    the Art of Learning
    where he emphasizes the importance of internalizing of the basics
    to the degree that it becomes your native language.

    Reply
  2. Cal — love your blog and work, but I have to admit this post really shocked me because there has been a long tradition in philosophy of trying to get at this very question, from Aristotle’s pioneering work on logic to Descartes’ Discourse on Method. And, since you’re mathematically inclined, it might delight you to know that Plato himself looked to the methods utilized by geometers for mathematical discovery as a model for articulating what we do in thinking in general. Plato was interested in how mathematical analysis lead to moments of deep comprehension (called “nous” in Greek); in turn, that deep comprehension was then re-organized into a synthesis:
    https://hsm.stackexchange.com/questions/5629/analysis-vs-synthesis-in-greek-mathematics?utm_medium=organic&utm_source=google_rich_qa&utm_campaign=google_rich_qa
    Sorry to lecture you here but philosophy is my bread and butter.

    Reply
  3. This book has a very useful discussion of focussed vs diffused thinking in learning science that may be worth discussing.
    https://barbaraoakley.com/books/a-mind-for-numbers/

    The eureka moment can come when you are relaxed (diffused) after concentrated effort on a problem (the incubation period in the literature on learning). In short, I don’t that there is one way to think deeply to move a problem ahead.

    Reply
  4. Aha!: The Moments of Insight that Shape Our World by William B Irvine is a good book on the topic of insights, how they come about, the role of subconscious mind, and the prerequisite of multiple attempts made by the conscious mind etc.

    Reply
  5. Hi Cal
    In a podcast interview for Straight A in Optimise, you mentioned that you systematically did trial and error to find the best study methods that suited your situation, and it seems that you’ve done this as well when creating a system that allowed you to master certain skills in a limited time period. I’d like to ask, how do you create such systems from scratch?

    Reply
  6. I couldn’t agree more with the last paragraph. Our culture see the eureka moment as something only related to “geniuses”; of course that they don’t tend to take in mind all the hard and deep work behind the moment.

    Indeed, this post is the right place to share one of my all time favourite quotes:

    “Chance favors the prepared mind.”
    -Louis Pasteur

    Reply
  7. Hi Cal,

    There was a post “Are You Using Social Media or Social Media Using You” & then another on “Brain Hacking and Sean Parker on building FB”, I found something that adds to that. Hope you and other out here find it useful.
    This video has formed FB employees including Sean Parker telling the deep dark things about FB itself.

    https://www.youtube.com/watch?v=39RS3XbT2pU

    Reply
  8. It’s a little bit complicated but really interesting.
    Learning, and reaching a better understanding something that will lead us to real changes.
    A.I. it’s simply a new frontier will be used to really have a better life for all the people?

    Reply
  9. How wonderful to hear what intelligent people are thinking. Refreshing and motivating, a great antidote to all the nonsense out there.

    Reply
  10. The best researchers in my field of theoretical computer science, for example, tend to be those who have the drive, brain power and flexibility to read other peoples’ papers constantly to spark a new recombination or extension that advances the field — an exhausting process. — From the article

    That’s precisely why I left academia. I got sick of reading poorly written, poorly reasoned tracts just so I’d have a set of references for my own. At least 90% of everything in academic journals is dull, pedestrian crap. I don’t see how it could inspire anyone to greater creative thinking. (Sorry….)

    Reply
  11. to really make big progress in science you also need times when I can be very focused ….see anything wrong here heavy thinker?

    Reply
  12. The missing link in this discussion is, humans have a soul. A soul is “who” a person is, and is separate from the brain. These scientists seem to be approaching the problem of computer learning (or truthfully AI) from an evolutionary baseline. Wrong assumptions end with wrong conclusions.
    When a person dies, their consciousness (soul) leaves the body along with their life force (spirit) and goes on to the afterlife without the brain (computer).

    Reply
  13. The subconscious: Brilliant people tend to ignore it because they feel as though they have it under control. But it will shape the directions they take in their creative thinking.

    There is a lot of missing data about this. It would change the world a lot if more people were aware of how insidious the subconscious can be, and knew how to handle it.

    A mastery of the basics is obviously necessary in doing things: The singer and his voice; the photographer and her camera; the coder and his IDE. But I think this is also true at higher more conceptual levels. It’s hard because so few ever achieve it, and if they do, they rarely write it down in a way that others could understand it.

    Reply
  14. You’ve changed my life when I saw your video on active recall in Texas 3 years ago. This one thing that you said changed how I approached everything that required learning from chess to learning math. You said “If you’re not actively explaining something without looking at your notes, YOU’RE NOT REALLY LEARNING.” I relied way too much on rereading my notes or someone else’s notes and after reading them, often times I would feel that I “understand” By putting your quote into practice I have realized that I can use “explaining out loud” as a litmus test to see if I really understand what I say I understand and do this in front of an mirror and I realized that I actually don’t understand what I say I understand. So basically you made me realize how clueless I really am and that’s the best thing that has happened to me. I’m now actively explaining things but I still need to work on simplifying my explanation for every subject.

    Reply

Leave a Comment