darb.ketyov.com

Caveat lector: This blog is where I try out new ideas. I will often be wrong, but that's the point.

Home | Personal | Entertainment | Professional | Publications | Blog

Search Archive

Loading...

26.12.11

A New Model for Scientific Publishing

There's a new paper out in Frontiers in Computational Neuroscience that is relavant to my interests. The paper is by Dwight Kravitz and Chris Baker from the NIMH and is titled "Toward a new model of scientific publishing: discussion and a proposal".

About two weeks ago Dwight emailed me his paper saying that he'd read a post I'd written last month for the Scientific American guest blog called "What is peer review for?" (you can check out my interview on Skeptically Speaking on this topic as well).

After reading Dwight's paper I want to make sure it gets as much exposure as possible. I can't it justice because it's so well-written and clear. But before I shuffle you off to read it I wanted to highlight their proposed system and ask you all what you think.

What are the barriers to instantiating their proposed system?

In the SciAm piece I concluded by saying:

But the current system of journals, editors who act as gatekeepers, one to three anonymous peer-reviewers, and so on is an outdated system built before technology provided better, more dynamic alternatives.

Why do scientists-–the heralds of exploration and new ideas in our society–-settle for such a sub-optimal system that is nearly 350 years old?

We can–-we should-–do better.

Well, it looks like Kravitz and Baker put a lot more thought into this problem than I and they've come up with an incredibly novel alternative system to peer review.

They nail it. There's almost nothing that I disagree with.

I love this paper.

They succeed here not because of their criticisms--which abound in the sciences--but rather because of their inclusion of a viable, creative, intelligent solution that addresses problems of motivation, utility, practicality, and even finance for an alternative model for peer review and scientific publication.

They begin by describing the current peer review system in the context of the neurosciences. They have an amusing graph that highlights the 17 levels of hell that is the peer review process loop. These guys crack me up.

"In the case of a rejection the Authors generally proceed to submit the paper to a different journal, beginning a journal loop bounded only by the number of journals available and the dignity of the Authors."

(click to enlarge)

Before I outline what their alternative proposal is, I want to highlight some of the problems regarding the costs and problems of the current system that Kravitz and Baker identify.

...what is striking is less the average amount of time [it takes to publish a paper], which is quite long, but more its unpredictability. In total, each paper was under review for an average of 122 days but with a minimum of 31 days and a maximum of 321. The average time between the first submission and acceptance, including time for revisions by the authors was 221 days (range: 31–533)...

...Beyond the costs of actually performing the research and preparing the first draft of the manuscript, it costs the field of neuroscience, and ultimately the funding agencies, approximately $4370 per paper and $9.2 million over the approximately 2100 neuroscience papers published last year. This excludes the substantial expense of the journal subscriptions required to actually read the research the field produces and the unquantifiable cost of the publishing lag (221 days) and the uncertainty incurred by that delay...

...Authors are incentivized to highlight the novelty of a result, often to the detriment of linking it with the previous literature or overarching theoretical frameworks. Worse still, the novelty constraint disincentives even performing incremental research or replications, as they cost just as much as running novel studies and will likely not be published in high-tier journals.

Okay, so what is their alternative model?

(click to enlarge)

It breaks down like this:

* No more editors as gate-keepers ("Their purpose is to serve the interests of the journal as a business and not the interests of Authors").
* Publication is guaranteed, so no more concern over "novelty".
* Editors instead coordinate the process and ensure that double-blind anonymity is maintained.
* Reviews are passed on to the authors as part of a "pre-reception" process which allows the authors to revise or retract their work before making it publicly available.
* Once public, the post-publication review process begins.
* An elected Editorial Board acts as an initial rating and classification service to put the paper in context.
* Members of the Board are financially incentivized... but the money doesn't go into their pockets, rather it can be put into their own research fund coffers.
* Papers are put into a forum wherein members of that forum can ask questions and offer follow-up suggestions.
* Forums provide a more living, dynamic quality to papers, as well as metrics for each manuscript.
* With better metadata for papers, ads could be more targeted by paper topic (no more ads for PCRs in cog neuro papers, for example).
* Kravitz and Baker note that something like a Facebook page for each paper could serve this purpose.

The Kravitz-Baker system saves money and time wasted by the needless "walking down the impact factor ladder" that usually occurs.

They address a lot more issues beyond what I've described above. They note that a common counter-argument to double-blind review, for example, is that a reviewer can "often guess" who the authors of the paper they're reviewing are because sub-fields are so small. But Kravitz and Baker point out that "the identity of the Authors might be guessed by the Reviewers, any ambiguity should act to reduce this bias". This seems so obvious.

The Kravitz-Baker system seems really well thought out, and one I'd love to see in place. But I'm worried I'm missing some critical fault here.

Anyone see any glaring issues?

ResearchBlogging.orgKravitz, D., & Baker, C. (2011). Toward a New Model of Scientific Publishing: Discussion and a Proposal Frontiers in Computational Neuroscience, 5 DOI: 10.3389/fncom.2011.00055

23.12.11

Am I a scientist?

Over on Quora, someone asked me to answer the question "Am I a scientist?"

They gave the following details:

I am in the process of getting my PhD. I spend every day doing research. I "do" science. Can I put on my business card, "John Smith, Scientist"? Do I have to wait until I have the PhD in hand? Or until I'm a candidate?

I took a quick crack at it, but I'd love to hear what other people think. Here's my answer in full:

*****

There are two issues here: the first is one of credentials and the second is one of societal interpretation.

"Back off man..."

In the case of credentials, there is no exam, or class, or quiz, or whatever that one needs to pass in order to become a "scientist". There is no "science" credentialing system. Certainly if you are a PhD researcher working at a scientific research facility then you are a scientist. But so are all of that person's subordinates, who may or may not hold a PhD or even a degree at all!

Are Diederik Stapel and Marc Hauser still scientists? They hold PhDs and conducted research, but both were caught falsifying and/or fabricating data. That's certainly not scientific!

The second issue is one of societal interpretation. If you put on your business card "SCIENTIST", that gives the person reading the card the impression that you are currently a practicing researcher or theoretician. If you are not such, then you are being duplicitous and should not "advertise" yourself as a scientist. Not because you're not a scientist, but because you're sending a signal that isn't entirely true.

In your case, you are a PhD student. You are doing research (I presume); therefore you are a scientist.

That said, I believe you'd be better off putting "John Smith / Ph.D. candidate, awesomeology / University of Very Impressive" on your business card, as it is accurate, sends a signal that you're a "dedicated" scientist (i.e., working toward a PhD) and that all will carry the baggage of "being a scientist" for you, without the need to explicitly state it.

Now for my totally biased opinion: just calling yourself a "scientist" (without explicating what field of science you're specializing in) comes across as a little sleazy, like you're taking advantage of a title to pull one over on people. It's like people who constantly refer to themselves as "doctor"...

(image source from an older post of mine)

8.12.11

Are toes pretty or ugly?

Another answer to a Quora question. Best. Blog-fodder. Ever.

What might cause the split between people who think toes are ugly and toes are pretty?

People have different experiences that lead to different behaviors. But it's not like my past experience in thinking some women are ugly causes me to think all women are ugly. Why should that be the case for any aesthetic experience?

Thankfully the questioner gave me an out by asking what might cause some people to find toes pretty while others think they're ugly.

I'm going to use that to my advantage to give a completely unsubstantiated, just-so answer that's too cool for me to ignore. If more scientific evidence comes out on this topic I'll try to remember to adjust my answer accordingly.

Short answer: toes and penii might be closely related (neurologically speaking). For more science (SCIENCE!) read on.

In my answer to Why can't I control my individual toes? (two toe-neuroscience answers?! o_O I'm not even into toes!) I introduced the motor homunculus:


This guy's body parts are distorted such that the size of a body part is proportional to the area in the primary motor cortex that is dedicated to representing that part. This was first determined by Wilder Penfield by stimulating people's brains and mapping the motor responses of the body.

Just behind the primary motor cortex (blue in the figure below) is the primary somatosensory cortex (red). The somatosensory cortex is the final common pathway for all incoming touch sensations of the body (pain, light touch, etc.)


The representation here is similar the motor cortex and mirrors it quite closely. Save one pretty striking exception.

Meet the (male) somatosensory homunculus:


If you're up for it, here's the uncensored, possibly NOT WORK SAFE version (if your workplace hates science).

The first thing you'll notice is that the representation of our toes is much bigger on the somatosensory compared to the motor homunculus. (That was the first thing you noticed, right?) This means our toes are given a lot of brain area in the somatosensory cortex, which means we have relatively more sensitivity in our toes than, say, an equivalent area on our shins.

So lets take a look at the somatosensory map to see what the layout of body parts looks like on the brain:


Check out the locations of the toes there on the right. At the top, where the butt is, is the top of the brain. This image represents only one half of the brain, so the butt is actually at the top center, and right across from it, in the other half of the brain, the other half of your butt is represented.

This means that the toes are actually represented on the medial surface, squished between the two hemispheres of the brain (with each hemisphere having a representation of the toes on the opposite side of the body).

See what body part is represented right next to the toes, though!?

GENITALIA! Yay! I'm so close to actually answering the question now!

(See Why does the writing style of most PhDs on Quora appear to be long-winded and poorly structured?, bro.)

Here's the theory put forth by UCSD neuroscience rockstar VS Ramachandran:

In some people, neurons coding for or representing genetical sensation are "cross-wired" with neurons representing toes and feet. This cross-talk may give rise to the sexual associations of toes and feet.

This is far from proved, but it makes for a nice story. Ramachandran has done some clever experiments to test his theories about how neuronal "cross-wiring" gives rise to certain behavioral phenomena, such as synesthesia, so it's not a totally out-there hypothesis.

Ramachandran relates an amusing story of one of his patients about this topic:

The next day the phone rang again. This time it was an engineer from Arkansas.

"Is this Dr. Ramachandran?"

"Yes."

"You know, I read about your work in the newspaper, and it's really exciting. I lost my leg below the knee about two months ago but there's still something I don't understand. I'd like your advice."

"What's that?"

"Well, I feel a little embarrassed to tell you this."

"Doctor, every time I have sexual intercourse, I experience sensations in my phantom foot. How do you explain that? My doctor said it doesn't make sense."

"Look," I said. "One possibility is that the genitals are right next to the foot in the body's brain maps. Don't worry about it."

He laughed nervously. "All that's fine, doctor. But you still don't understand. You see, I actually experience my orgasm in my foot. And therefore it's much bigger than it used to be because it's no longer confined to my genitals."

It may be that neuroplasticity after this patient's limb loss induced communication between his foot and genital sensory neurons. This observation lends some support to Ramachandran's toe/brain/penis hypothesis.

For more on cool stuff related to neuroplasticity, see my answers to Can brain trauma cause cognitive enhancement? and When parts of the brain are removed during surgery, is it possible for the remaining brain tissue to expand into the available space?

For more reading on sex, brains, and homunculi, check out the neurocritic who's covered recent neuroscience research looking at the somatosensory representation of circumcised v uncircumcised male penises, the representation of the female homunculus, and the neuroscientific attempt to find the clitoris.

ResearchBlogging.orgHubbard EM, Brang D, & Ramachandran VS (2011). The cross-activation theory at 10. Journal of Neuropsychology, 5 (2), 152-77 PMID: 21923784

7.12.11

SciPle.org interview

Lots of interviews this week!

Recently I presented some of the latest developments in my brainSCANr project at the Society for Neuroscience conference.

At my poster I was approached by SciPle.org, and Leonardo Restivo asked if I would do an interview with them. Well the interview's been posted and my answers are below. They're more personal than I was expecting, but I try not to shy away from personal questions (even if the answers can be uncomfortable).

Check Sci.Ple out; follow them on Twitter. They've got some really cool ideas that I'd love to see work out.

Sci.Ple: What is your background?
Technically I began my undergraduate career at the University of Southern California as a physics major. I grew up in San Diego with a lot of clear night skies. I wanted to be an astrophysicist. But that was not to be my path, and I ended up doing a degree in psychology while taking a host of philosophy and cognitive and computer science courses. I then worked as a research associate at UCLA for Edythe London; I was the PET scanner operator and radioactive pee cleaner (that's another story). In 2010 I completed my PhD in neuroscience at UC Berkeley under the mentorship of Robert Knight, the institute's director at the time. I'm currently a post-doctoral fellow at UCSF working with Adam Gazzaley.

Sci.Ple: Among your published papers, which one is your favorite?
Okay, I'm cheating a bit since it's not actually published yet, but it's close so I'm just going to act like it's published and hope that talking about it doesn't jinx the whole process. My favorite paper is one I co-wrote with my wife Jessica and is titled "Automated Cognome Construction and Semi-automated Hypothesis Generation". If I play by the rules of the question, then I'll go with "Hemicraniectomy: A new model for human electrophysiology with high spatio-temporal resolution" (Voytek et al., J Cogn Neurosci 2010).

Sci.Ple: Why is it your favorite?
Honestly I believe that the "Semi-automated Hypothesis Generation" paper is a Big Deal. We're text-mining the abstracts of millions of peer-reviewed neuroscience papers to try and make some sense out of how neuroscientific concepts interrelate. Then we're going one step further to see if we can find statistical "holes" in the literature… we're literally trying to (semi)automate one aspect of the scientific method: hypothesis generation.

Sci.Ple: What was the most challenging part of this paper?
The first spark of the idea for this paper came about during a panel I was on at a cognitive science conference in 2010. In response to a question from an audience member, I said something like, "the scientific literature is smarter than we are; many basic facts about brain function are probably already known, but we suck at synthesizing it all." Trying to prove that statement was the foundation for his paper. In order to write the paper I had to learn about text-mining, some graph theory, etc. It's totally outside of my comfort zone, but I think it's too cool to stress out about it not being "perfect".

Sci.Ple: What drives you in your day-to-day job?
Honestly? People literally pay to me to think about cool shit. That's my whole job. I get to say "I wonder if anyone has done this before," and then go out, run some experiments, and then possibly learn something that no one has ever known before. I've worked on a loading dock carrying heavy things onto trucks for up to 16 hours at a time. I've worked at a motel where my job was to go room to room and collect all of the... soiled... linens. So I guess the simplest answer to what drives me is a mix of awe and perspective. That's what keeps me going to work every day.
 
Sci.Ple: What is the most exciting part of your job?
Talking to other people. Collaborating and combining the accumulated knowledge of multiple brains in new ways to tackle hard problems. That's amazing. It's humbling and inspiring to see brilliant people's minds work.

Sci.Ple: The least exciting?
Paper formatting and data munging.
 
Sci.Ple: Name a scientist whose research inspires you
Reed Richards. That guy seems to be able to produce an endless stream of amazing, breakthrough ideas. Oh. Seriously? I've got a huge amount of respect for pre-digital scientists. Lots of them were going out and just trying everything. Sever the nerves in your own arm? Why not?! Cover yourself in varnish to prove that the skin does something important? Who cares if you almost died? You were right! I don't think I have the… fortitude… to do a lot of what those folks did--from the crazy self-experimentation to the drudgery of pre-digital writing and researching--I can't help but respect that.

Sci.Ple: What are the next frontiers in neuroscience?
Information integration. How do you go from a neuron giving off an action potential to a thought? From a neurotransmitter binding to a receptor to art? Imagine a Venn diagram that contains medicine, biology, genetics, psychology, philosophy, engineering, computer science, mathematics. That's neuroscience. Knowledge in each of these fields is being slowly accumulated (almost always separately) by doctors, biologists, geneticists, psychologists, philosophers, engineers, computer scientists, and mathematicians. We need more cross-disciplinary work.
 
Sci.Ple: Why science?
Why anything? My life, like almost everyone else's, has consisted of a series of half-blind stumbling steps. I got lucky. That said, I'll take the minor frustration of getting a paper rejected or having someone disagree with me over "soiled" towels and sheets any day.

Sci.Ple: If not science?
I'd work as a bartender at a pub. Or try and run my own. I love meeting and talking with people, and few places are better for than that a nice local watering hole.

Sci.Ple: Why?
"What happens to a werewolf on the moon?" I read that online the other day and it still cracks me up.

4.12.11

Skeptically Speaking radio interview

Last month I wrote a piece about peer review for Scientific American.

Shortly thereafter I was contacted by Desiree Schell from the Skeptically Speaking radio show about doing a short interview on the topic. I love this podcast (it's one four I listen to after Science Friday, Radiolab, and TED talks), so I was pretty excited.

And--according to my wife--it was actually interesting. So yay to not being a boring knob. The interview starts with a discussion about faster than light neutrinos; I pop in around minute 43.

Here's the episode.

Anyway, if you haven't checked this show out, please do so. Their archives have great interviews with scientists and general science geeks such as Adam Savage, XKCD's Randall Munroe, and many others.