darb.ketyov.com

Caveat lector: This blog is where I try out new ideas. I will often be wrong, but that's the point.

Home | Personal | Entertainment | Professional | Publications | Blog

Search Archive

26.11.10

Cargo cults of the brain

During World War II, Japanese and American troops operated in large sections of Melanesia. Both sides brought in huge amounts of food, equipment, and supplies for their troops, though by the end of the war the US occupied this region. These islands were inhabited by small indigenous tribes who had never seen such abundance.

After World War II the US abandoned their posts, stopped bringing in supplies, and left the islands. Over time, certain members of the tribe began mimicking the behaviors of the soldiers they had previously seen bringing in vast amounts of wealth. According to Wikipedia,

Cult behaviors usually involved mimicking the day to day activities and dress styles of US soldiers, such as performing parade ground drills with wooden or salvaged rifles. The islanders carved headphones from wood and wore them while sitting in fabricated control towers. They waved the landing signals while standing on the runways. They lit signal fires and torches to light up runways and lighthouses.

In a form of sympathetic magic, many built life-size replicas of airplanes out of straw and cut new military-style landing strips out of the jungle, hoping to attract more airplanes. The cult members thought that the foreigners had some special connection to the deities and ancestors of the natives, who were the only beings powerful enough to produce such riches.

Bradley Voytek cargo cult airplane


The tribes didn't understand how the US troops has access to so much wealth. To try and get at that wealth themselves, they simply mimicked the behavior of the troops. Obviously this is a fallacy: mimicking the behavior of a thing is not the same as understanding the thing. To put it another way:

The map is not the territory.

In 2010, we have major neuroscientific endeavors such as the Blue Brain Project. According to this group,

The facility has been used to build the first model of the neocortical column, which consists of 10,000 3D digitizations of real neurons that are populated with model ion channels constrained by the genetic makeup of over 200 different types of neurons. A parallel supercomputer is used to build the model and perform the experiments so that the behavior of the tissue can be predicted through simulations.

Bradley Voytek blue brain project


Maps? Territories? As I said in my old post on the subject,

To think that modeling a bunch of neurons digitally is akin to a thinking, evolved, conscious, aware human brain is like thinking that by soldering together a couple of million transistors in a "Apple-like fashion" will give you a working MacbookPro.

Now, I swear I'm not picking specifically on Blue Brain. I love Blue Brain. I want Blue Brain to succeed. I'm a huge sci-fi nerd! I want my cool brain-computer interfaces, AI, etc. When I wrote a skeptical post about the Blue Brain Project back in July, there were a few challenging comments written in response. However, my critiques of this project are certainly not the only ones.

When the project first started up in 2005, Nature ran a brief news piece on the Blue Brain Project wherein they discussed some of the very same concerns I voiced in my post.

This is an ambitious project that is bound to fail," says Terry Sejnowski of the Salk Institute for Biological Studies in San Diego, California. "We are still far from understanding enough about the brain to build a detailed realistic model."

Neuroscientists say that too little is known about the structure of the network connecting cortical cells, for example. They add that a truly realistic model would have to incorporate molecular activity in the regions where neurons connect, a level of detail that is currently beyond the Blue Brain Project.

Right. Well, at the time of my posting, I hadn't read that, but that was pretty much what I said:

...[A]t this point we honestly just don't know enough about how all the pieces play together to give rise to our cognition. I think the neurosciences right now are where physics was in the early 1900s. A bunch of people thought Newtonian mechanics could explain everything. Turns out, the physical universe is much more complicated than that....

...[W]e know a lot about the biology of the neuron. Similarly, computational modeling has gotten very sophisticated. When researchers build computational models incorporating known biology, they call it a "biologically-plausible" model. I think we're still stuck in the Newtonian mechanics period of neuroscience, and we're just now segueing into the more complicated "oh my god this stuff is harder than we thought!" part of our science.

However, people seem to think that the fact that I don't believe that the Blue Brain Project, as currently instantiated, will give us a human brain model somehow translates to me thinking that the Project shouldn't be done. As I said in the comments to my post,

Blue Brain's an excellent step in the right direction. However the people selling it are over-hyping what we do know in neuroscience.

There's a difference between the practice of science and the salesmanship of science. For the former, failure is a critical component!

This month, in Nature, Melanie Stefan wrote an excellent piece on how individual scientists should emphasize their failures, a practice which I myself have put into play in my own CV!

I embrace failure! And Blue Brain will fail! And then we scientists will address those failures, highlight and repair the faults, embrace the successes, and iterate forward. And future Blue Brain may very well succeed!

But that's not my issue.

My main issue is the salesmanship. The grantsmanship. On their own site, Blue Brain claims that it is, "A Novel Tool for Drug Discovery for Brain Disorders" that will, "...provide a concrete foundation to explore the cellular and synaptic bases of a wide spectrum of neurological and psychiatric diseases."

This is called "over-hyping", and it happens all the time. Of course, this isn't a problem just with Blue Brain, or even in science. Anyone who has interviewed someone for a job, been on a date, or basically lived in the western world will be quite familiar with this phenomenon.

But, when it comes to science especially, over-hyping needs to be reigned in. It's a problem that is endemic to the very system in which modern science operates.

Now the new hotness is "connectomics", a research path for which I am a very strong advocate! As my friend Josh says, neuroanatomy is the RULES! If you don't know the anatomy then you can't say much about the brain!

Bradley Voytek human connectome project


Well, this week, Nature Neuroscience takes the over-hyping of the Human Connectom Project to task. (By the way, tons of love for the Nature Publishing Group in this post, apparently. Of course, being NPG, all of their links will be paywalled and thus inaccessible to many readers... ::sigh::)

Anyway, the piece, "A critical look at connectomics", astutely points out that,

Local connections in brain regions, which are roughly 80% of the connections in the cerebral cortex, are invisible to [current] imaging methods. Thus... the Human Connectome Project will necessarily provide us with partial and probabilistic data.

This is similar to the issue with the Blue Brain Project wherein they're currently (I believe) only modeling neocortical columns (for now). This represents fewer than half of the neurons in the human brain, fewer than 5% of the total cells in the brain, and who knows what portion of cognition (if it even makes sense to talk about "cognition" outside of a full brain). Of course, given my love for subcortical brain regions (see my PNAS paper), I'm a bit biased, but non-cortical brain areas really can't be ignored when you're talking about understanding brain disorders!

The editors at Nature Neuroscience go on to say,

It's tempting to sell the Human Connectome Project, and connectomics in general, as directly relevant to disease, particularly given the public money invested. However, given the challenges that this field is facing, it seems ill-advised to present connectomics as providing immediate answers for disease when it is clear that this is a long-term goal that will require the continued support and collaboration of the neuroscience community and the tax-paying public.

To translate: as scientists, the hyping that we do to get grants commits us to a message that may very well be detrimental to the very scientific endeavors that we love so much that we've dedicated our lives to pursuing them. Blue Brain does this. I've done this. Anyone who's written a grant has done this. And we need to stop.

So this post is to affirm my commitment to reigning in the hype. We can be exciting and relevant, without blowing smoke up the public's ass.

[EDIT: A friend pointed out to me that this post is similar in concept to Feynman's "cargo cult science" idea. While I wasn't consciously aware of this prior to writing this post, I still feel compelled to point that out in case I was unconsciously building off of it. Plus, Feynman is awesome and I love that phrase now and will be using it often, I'm sure, much to the annoyance of my wife, friends, and colleagues.]

11 comments:

  1. Therein lies the downright stupidity of the Research Councils' requirement to write about the "impact" of your research before you have even started it. That doesn't just encourage dishonest hype. It makes it compulsory.

    ReplyDelete
  2. Great post Brad. Here's Philip Gerrans on similar themes

    http://www.timeshighereducation.co.uk/story.asp?storycode=407277

    ReplyDelete
  3. Thanks David and Jon!

    David: That's *absurd*. We're scientists; we're not to be prognosticators of outcomes and impact! The whole point of research is to address the unknown and expand the known, not to imagine what the future will be. Also, it's nice to see you're on Twitter, too. I've been following Improbable Science for a while now.

    Jon: Very cool. Thanks for sharing. That's an enjoyable article (though went on longer than necessary, I think). It's a very similar (and more modern) analogy than what I was making, too.

    ReplyDelete
  4. While I both participate and am critical of connectivity research, I'm not sure I buy Nature Neuro's examples of criticism. I think I found the (unreferenced) press release which is Nature Neuro's only specific example of over-hyped research, but it seems like they're quote mining.
    http://www.ninds.nih.gov/news_and_events/news_articles/pressrelease_blueprint_connectome_launch.htm

    The RFA press release is making clear that local connections are important, but that past research portfolios have been biased towards these connections and this is a large initiative to balance out how the long-distance connections fit in.

    I also disagree with the direct clinical applicability. Opening up any journal or walking around any neuro conference, you see dozens of groups presenting connectivity data showing clinical differences. Calling things like autism or schizophrenia connectivity disorders is heavily based on this research. There are currently diagnostics and treatments attempting to use these methods as metrics. The foundations are often shaky, which is one a big goal of the grant IS to get huge amounts of normal data to better understand healthy variation and to design methods to share the data and optimal analysis methods.

    There is over-hyping, but much happens in the lay press or with select quotes from individuals none of which the editorial was willing to point out... merely a vague consortium of important people are over-hyping.

    ReplyDelete
  5. Anonymous12:31

    I had a brief and frustrating exchange regarding Blue Brain with Steve Zara in his comment section last August. I did try to raise some of these objections, but the conversation just seemed to go in tight circles and I after three posts I abandoned it.

    Zara continues to insist that sub-cellular processes pose no significant hurdle to modeling neurons--for some reason, many who would surely scoff at the prospects for accurately simulating an amoeba or even a human liver cell in the near future nonetheless see little difficulty where brain cells are concerned.

    ReplyDelete
  6. Anonymous23:53

    Terry once mentioned that his comment about the Blue Brain is only half quoted by Nature. His argument was that his estimation of the computational power to achieve their goal exceeds what they were planning to use. That is why he said that the project is bounded to fail, but he said that he had commented that we will learn tremendously many things in the process, and this had not been quoted.

    I agree that the public relations publish ridiculous hyperboles, but I wound not underestimate its importance since it is the first serious attempt to make a "theory" in neuroscience --- not like those puny speculations that biologists call "theories". If this attempt hits deadlock, we all have to seriously think about what we are missing to study the cortex.

    ReplyDelete
  7. The idea that Blue Brain is going to reveal new drugs any time soon is silly. for one thing because there are few, if any, disorders which primarily affect the neocortex. Maybe some kinds of epilepsy.

    The idea that it might reveal very interesting things about the cortex is much less silly. But as has been mentioned... saying that doesn't get grants.

    ReplyDelete
  8. Dan:

    Yeah, that's weird that Nature Neuro doesn't reference *where* they're getting this information from.

    And of course, getting grants is critical to trying things out. And most of those hypotheses will be wrong. But that's what research is for and that's okay!

    I liked this quote from neurocritic, by the way:

    "Observation from SFN: Functional connectivity in the default network correlates with everything under the sun and is associated with every known neurological condition."

    http://neurocritic.blogspot.com/2010/11/random-observations-from-sfn-2010.html#c6817284447435617594

    ReplyDelete
  9. Anonymous 1:

    Yeah, that doesn't make any sense... when I got into an exchange in my last post on this topic I asked how they plan on modeling non-local neuronal communication such as NO, which diffuses long distances but still acts as a neurotransmitter.

    ReplyDelete
  10. Anonymous 2:

    That's unfortunate. Because he's right... there's still a lot to learn, even if they're not *actually* "simulating a human brain".

    ReplyDelete
  11. Neuroskeptic:

    Ha! I hadn't even considered the fact that very *few* disorders are cortical.

    ReplyDelete