darb.ketyov.com

Caveat lector: This blog is where I try out new ideas. I will often be wrong, but that's the point.

Home | Personal | Entertainment | Professional | Publications | Blog

Search Archive

Loading...

21.3.14

A decade of reverse-engineering the brain

Salesmanship trumps science. Every. Single. Time.

The big news in the tech world today is the superstar team-up of Elon Musk, Mark Zuckerberg, and Ashton Kutcher investing $40M in Vicarious, whose aim is to, "[t]ranslate the neocortex into computer code". Because then “you have a computer that thinks like a person," according to Vicarious co-founder Scott Phoenix. “Except it doesn’t have to eat or sleep.”

I took at look at this mystery team of neuroscientists who've secretly reverse-engineered how the human brain works and, according to the Vicarious team page, the scientific talent (and I assume lead) is Dileep George.

George was formerly the CTO of Numenta, the company that was spun out of Palm founder Jeff Hawkins' book On Intelligence (which is a fine book with a neat theory, by the way).

Hawkins founded the Redwood Neuroscience Institute which eventually was absorbed into UC Berkeley as the Redwood Center for Theoretical Neuroscience. This was all happening right when I began my PhD at Berkeley.

In 2004.

George gave a talk at the Accelerating Change conference in 2005, the abstract of which reads:
We are at a juncture where great progress has been made in the understanding of the workings of the human neocortex. This gives us a unique opportunity to convert this knowledge into a technology that will solve important problems in computer vision, artificial intelligence, robotics and machine learning. In this talk, based on joint work with Jeff Hawkins, I will describe the state of our understanding of neocortical function and the role Numenta is playing in the development of a new technology modeled after the neocortex.
My question is, how is Vicarious different? What's changed in the last 9 or 10 years or so? Because the high-level press release stuff sounds exactly the same as the Numenta stuff from a decade ago.

What happened to Numenta's lofty aims?

They're now called "Grok" and, according to their about page:
Grok, formerly known as Numenta, builds solutions that help companies automatically and intelligently act on their data. Grok’s technology and product platform are based on biologically inspired machine learning technology first described in co-founder Jeff Hawkin's book, On Intelligence. Grok ingests data streams and creates actionable predictions in real time. Grok's automated modeling and continuous learning capabilities makes it uniquely suited to drive intelligent action from fast data.
George did some amazing computational neuroscience research at Numenta. But for all the talk about how slow academia is, you'd think after ten years and tens (hundreds?) of millions of dollars in spent in the fast-paced world of private industry, the sales pitch would have changed by now.

The Blue Brain Project is nearing the end of its first decade as well. And, again, there's some great work coming out of these places, but I cannot overstate my frustration at the hype-to-deliverables ratio of these organizations.

Granted, I wasn't in the meetings. Maybe a lot has changed, but none of that change is making its way out to anywhere where the rest of us can see it.

Having watched this stuff for a decade now, the grand promises have not been delivered on. It's clear to me that VCs need some skeptics on their advisory teams. Any neuroscientist and/or machine learning researcher in that meeting would certainly ask:

"What's different?"

14.3.14

The Passion Trap

The first email I ever sent was to Stephen Hawking. I sent the email in the spring of 1998 when I was 16 years old from a computer at my high school (because I didn't have internet at home) using a friend's AOL account. I had just finished reading Hawking's A Brief History of Time and knew that I wanted to be an astrophysicist (or a cosmologist). I emailed Hawking to tell him how much of an inspiration he was to me and how passionate I was about physics.

Passion. Follow your passion. For those of us lucky to have choices in our life trajectory we're bombarded by advice to follow our passions. Chase your dreams. Go to culinary school. Major in whatever you love. Drop out and start a company! Listen to no one, just follow your heart!

A quick look over at Amazon suggests this is a lucrative bit of advice.

But (and I'm definitely not the first to say this) it's not the best advice.

When I was in high school I was a physics and math chauvinist. I saw psychology and the biological sciences as "soft". My love for physics was probably planted in part by this goofy book:


I've always been inclined toward the sciences, but that book lit a fire in me. It got my imagination going about what could be possible if enough smart people got together to work on a Big Idea. This creative aspect of science really drew me in and, I've come to realize, shaped my career.

Whenever anyone asked 10-year-old me what he wanted to be when he grew up, I'd answer "an astrophysicist". Yeah, I wasn't the coolest kid. But that spark stayed with me and I found some fun outlets. I spent a lot of time in high school playing video games, role playing games with friends, etc. All of the nerd-flavored creative outlets.

As for school, it was was easy and I coasted through.

Home life was… non-standard… so when I was given the opportunity to skip my senior year of high school to attend the University of Southern California I seized it. One late August night in 1998, at about 2 am, I called up my buddy Curtis and he drove me to Los Angeles to drop me off at college.

I immediately declared as a physics major and kept going with all the "advanced" versions of the courses. Around the same time I discovered that I enjoyed socializing and I made a lot of new friends. One part of my life was rewarding, the other was not, so I stopped going to classes. I did poorly, but I had a lot of fun doing it. My love for physics started waning due to the monotony of the work and the lack of wonder exhibited by the professionals I saw working in academic physics.

The only reason I didn't drop physics sooner was the fear that my physics friends would make fun of me for "going soft". And because I didn't know what else to do.

Physics was all I'd ever wanted to do. Physics was my passion.

There's that word again.

Becoming a astrophysicist was this grand ideal I'd built up for myself. It had become part of my identity. Once you start defining yourself by one thing—a political belief, religious affiliation, career, family, whatever—you lose identity to that thing. You reduce the number of paths to happiness and success and wrap your entire self around it.

To put it mildly: that can be unhealthy.

Modern psychological thinking generally breaks "passion" into two distinct subtypes. In their highly influential 2003 Journal of Personality and Social Psychology paper, Les Passions de l’Âme: On Obsessive and Harmonious Passion, Vallerand and colleagues differentiate harmonious passion (HP) from obsessive passion (OP):
Harmonious passion (HP) results from an autonomous internalization of the activity into the person’s identity. An autonomous internalization occurs when individuals have freely accepted the activity as important for them without any contingencies attached to it. This type of internalization produces a motivational force to engage in the activity willingly and engenders a sense of volition and personal endorsement about pursuing the activity. Individuals are not compelled to do the activity but rather they freely choose to do so. With this type of passion, the activity occupies a significant but not overpowering space in the person’s identity and is in harmony with other aspects of the person’s life. 
Obsessive passion (OP), by contrast, results from a controlled internalization of the activity into one’s identity. Such an internalization originates from intrapersonal and/or interpersonal pressure either because certain contingencies are attached to the activity such as feelings of social acceptance or self-esteem, or because the sense of excitement derived from activity engagement becomes uncontrollable. Thus, although individuals like the activity, they feel compelled to engage in it because of these internal contingencies that come to control them. They cannot help but to engage in the passionate activity. The passion must run its course as it controls the person. Because activity engagement is out of the person’s control, it eventually takes disproportionate space in the person’s identity and causes conflict with other activities in the person’s life. 
I hate to go all "Medical students' disease" here but this really seems to capture the gist of my personal physics passion struggle. Breaking out of that was very hard for me. It really felt like I was abandoning my identity. Or like I was lying to myself about who I am.

During my sophomore year I lived in a crazy place. One of my friends wanted to take a psych class and, because I had a free slot in my schedule and I had no idea what to do, I took that class with him. The classes I did attend were pretty cool. Dammit if it didn't turn out that people, and not just particles, are fascinating, too!

Fast forward one semester: I go to register for classes my junior year and find out that my grades had been too low for too long and I was basically kicked out of school. Long story short: I plead and begged, got a one-semester reprieve, got my shit together, and became a psychology major. I finished all the required courses in a semester.

I devoured the stuff.

At the time USC only had a cell/molecular biology major. No cognitive neuroscience. So I basically made my own major (though my final degree was in Psychology). I took C++ and Java classes, AI, Philosophy of Mind, Communication, etc.

I volunteered in a research lab as an RA and discovered that my ability to write code was a semi-magical skill because I could automate a lot of laborious manual jobs. I learned that I had a "knack" for approaching problems that way.

Really my interests as a doe-eyed wannabe cosmologist kid aren't that different from my doe-eyed adult neuroscientist self. My weird childhood, party-fueled and tumultuous college years, and crazy friends made me odd but kept me optimistic and protected me from being jaded. Ironically I now use a ton of math and physics in my neuroscience work.

Take that chauvinistic past-me.

Now, instead of asking "how are we all here, these tiny specs in the vast universe, pondering our origins?" I spend my days asking "how are we all here pondering our origins, we tiny specs in this vast universe?"

Ask yourself if you are harmoniously passionate, or obsessively, and if the answer is the latter, remember you are not your job, your belief, your class, your color, or your passion. To paraphrase a dear friend of mine: don't follow your passions, follow your competencies, and you might just find you enjoy doing something you're good at.

ResearchBlogging.org

Vallerand, R., Blanchard, C., Mageau, G., Koestner, R., Ratelle, C., Léonard, M., Gagné, M., & Marsolais, J. (2003). Les passions de l'âme: On obsessive and harmonious passion. Journal of Personality and Social Psychology, 85 (4), 756-767 DOI: 10.1037/0022-3514.85.4.756

3.3.14

Neuroscience, culture, and the public trust

Every year, the Society for Neuroscience holds a "Social Issues Roundtable" at their annual conference. Social issues in neuroscience are near and dear to me, and so this year I took a stab at submitting a proposal.

What better place to talk about these issues than at a conference of 35,000 neuroscientists?

Sadly the proposal (below) was rejected. The program had five speakers, consisting of:

  • Carl Zimmer (New York Times science writer extraordinaire)
  • Sally Satel (Psychiatrist and author of, most recently, Brainwashed: The Seductive Appeal of Mindless Neuroscience)
  • Vaughan Bell (Clinical psychologist, science writer, blogger at MindHacks)
  • Jeni Kubota (Neuroscientist studying stereotype and prejudice change)
  • David Higgins (Science fiction scholar, head of Science Fiction Literature (SF) for the International Association for the Fantastic in the Arts)

Note I did not include myself on the panel. I would have simply moderated.

Despite being rejected, I still think it's an important topic. So I'm looking into the possibility of hosting something anyway, either as a satellite to SfN, or at my university (UC San Diego).

If this sounds interesting or intriguing, please let me know!

Proposal

The Decade of the Brain. The $300M+ BRAIN Initiative. The public looks to neuroscience for answers about mental illness, cognitive decline, law, and disease. Society's expectations are shaped by media representations—from The Matrix to Malcolm Gladwell—which give an unrealistic view of both the certainty and capabilities of neuroscience. In this program we examine the effect of the dissonance between societal expectations and the nuances of scientific research on the public trust.

This roundtable aims to foster communication between media experts and creators, primary neuroscience researchers, mental health professionals, and culture studies researchers. This communication will focus on the bi-directional role between neuroscience research and the popular media and press, and how this affects the public trust. Specifically, there are three central themes we will explore:

  1. Why does neuroscientific research so readily capture the public attention?
  2. How do recurring themes of popular press accounts of neuroscientific research affect the public trust in the research? Examples of such themes include appeals to new research having "implications toward a cure for", e.g., autism, depression, anxiety, Alzheimer's, schizophrenia, etc. while never fulfilling those promises.
  3. How is neuroscientific research and funding affected by shifts in the public interest, from Matrix-like brain- computer interfaces to current trends in Big Data?

By addressing these issues and opening the dialog between neuroscientists and the media, we as a society can better understand how neuroscience is perceived in modern society and the role that we play as guardians of the public trust.

The 2013 BRAIN Initiative put neuroscience back into the forefront of the public’s awareness. Similarly, the release of the much-maligned DSM-5 and Thomas Insel's official statement on behalf of the NIMH on supporting RDoC have led to a resurgence in discussions surrounding our understanding of the biological basis for mental illness. Finally, neuroscience-based bestselling books abound: Grandin's ”The Autistic Brian", Ariely's "Predictably Irrational", and Kahneman's "Thinking, Fast and Slow".

30.1.14

Big data: What's it good for?

Recently I was interviewed for a piece in the Independent titled, "The number crunch: Will Big Data transform your life - or make it a misery?"

Part of this interview (my portion of which amusingly got truncated to "STUFF IS COOL") was around what "Big Data" is "for". Because what was included in the interview was shorter than what we talked about, I thought I'd use my own personal platform here to flesh it out a bit.

First, to get this out of the way, "Big Data" is literally just a lot of data. While it's more of a marketing term than anything, the implication is usually that you have so much data that you can't analyze all of the data at once because the amount of memory (RAM) it would take to hold the data in memory to process and analyze it is greater than the amount of available memory.

This means that analyses usually have to be done on random segments of data, which allows models to be built to compare against other parts of the data.

To break that down in simple words, let's say that Facebook wants to know which ads work best for people with college degrees. Let's say there are 200,000,000 Facebook users with college degrees, and they have been each served 100 ads. That's 20,000,000,000 events of interest, and each "event" (an ad being served) contains several data points (features) about the ad: what was the ad for? Did it have a picture in it? Was there a man or woman in the ad? How big was the ad? What was the most prominent color? Let's say for each ad there are 50 "features". This means you have 1,000,000,000,000 (one trillion) pieces of data to sort through. If each "piece" of data was only 100 bytes, you'd have about 93 GB of data to parse. That's pretty big (but still arguably not quite into "big data" territory), but you get the idea.

Your goal is to figure out which features are most effective in getting college grads to click ads. Maybe your first-pass model on a random sample of 1,000,000 users finds that ads with people in them that are 200x200 pixels big and about food get the most clicks. Now you have a "prediction model" for what college grads want, and you can then test that to see how well your prediction (based on the 1,000,000 college grads) holds up when you compare it to the other 199,000,000 college grads.

Now, for what it can do in "daily life", well, pretty much any company with a significant tech group (Google, Twitter, Facebook, any bank or financial institution, any communications and mobile service, energy, etc.) are doing this kind of thing. To serve ads, to improve their services, to predict future growth and demand needs, whatever. Relatively benign, boring, money-making stuff.

But what about other uses?

Google famously showed that they could predict flu outbreaks based upon when and where people were searching for flu-related terms:


There's the famous story about how Target's algorithms discovered a girl was pregnant.

Researchers are using Facebook statuses to look at how gender and age is affecting language use:


Doctors can look at what patients are writing about in online disease forums to try and get an idea of how off-label drug use affects certain diseases.

We can look at the evolution of language:


or the suppression of ideas:


We can look at how people move based on their cell phone use:


How money physically moves:


Or, like my work with Uber, their actual travel, and how various real world events (like the 2013 U.S. Federal Government Shutdown) affect the way people move around:


These are only the tip of the iceberg. 90% of the world's digital data was created in the last two years so we're just starting to figure out the possibilities. Note that in my cognition research I'm using a ton of data on peoples' behavior to try and infer how age, location, education, etc. affect our cognitive abilities. But those data aren't published or peer-reviewed yet, so it's not really appropriate to discuss quite yet. But the results are fascinating.

So yes, while the early focus of Big Data was essentially basic profit-driven advertising, one shouldn't hold onto the belief that that is all that it's good for.

Unfortunately, this is an extremely complex topic that sits at the intersection of personal freedom, privacy, industry, science, medicine, etc. The next ten years will be dominated (rightly so) by conversations surrounding data ownership rights and privacy. There's no reason that these kinds of analyses can't be done on anonymized data--so we shouldn't throw the baby out with the bathwater--but any scientists, researchers, or analysts should be mindful of these issues.

15.12.13

Brain Log? Collaborative academic neuroscience blog

Earlier today on Twitter I mentioned that I feel that academic neuroscience could use a collaborative, professional blog such as Language Log. This seemed to resonate with several people, but 140 characters isn't enough to explain why I feel this way.

First and foremost: for those of you not familiar with Language Log, it's a blog run by academic Linguists and other language professionals, including several journalists who cover language.

Off the top of my head, here are a few reasons why I think neuroscience could use a "Brain Log":

  • Language Log is collaborative, meaning it doesn't "belong" to any one person.
  • This minimizes one issue of a lot of personal blogs: personal "branding" and link-bait style posts. The focus is on solving problems rather than general interestingness.
  • Many neuroscience blogs are fantastic, and have higher level discussions (e.g., Neuroskeptic), but they're run on a professional site and thus are for-pay content. And they're still "branded" by their individual ownership, leaving further discussions at the discretion of the owner.
  • The scope of Language Log is more technical, which allows the people "in the trenches" to dig into details that the general public may not find very interesting, but young researchers, journalists, etc. may find very useful as a resource.
  • A collaborative blog allows for better two-way discussion between the primary scientists and journalists covering neuroscience (a very hot topic in the public interest right now).
  • This would allow journalists better access to broader scientific opinions, and to give them an easy way to reach out to academics.