Why don't academics discuss research before starting the work?!

Warning: this is a hopelessly idealistic proposal...

As far as I can tell, scientists of all types tend to be secretive about their work prior to publication. I guess the fear is that, without secrecy, another lab might steal the idea and publish it first. But this secrecy comes with several significant costs, one of which is the huge length of time between first having an idea and then getting feedback from peers. First you have your bright idea, then you do the necessary research (which might take years), then you write up a paper, then you submit the paper to a journal or conference and a while later you get your first bit of peer feedback.

This feels extraordinarily inefficient. If an idea is bad then it needs to fail fast (before resources have been expended). And if an idea needs improving then it's far better to improve the proposal before doing all the work rather than trying to retrofit a fix after the bulk of the work has been done.

How about this as an alternative:

When you first have an idea, you describe it in a few paragraphs and then submit it to an impartial website for comments from your peers. Comments might be along the lines of "Don't bother because Jo Blogs et al did something similar a few years ago" or "that idea is fundamentally flawed because of X" or "it's a crazy idea but it might just work: give it a shot; but to convince me you'll need to demonstrate Y" or "Great idea! I was thinking of doing something similar. Fancy collaborating?"

This way, you get very rapid feedback from your peers before you commit large quantities of time and, possibly, money. You also get some protection against people taking your ideas and claiming that they thought of them first (because your ideas have been published; albeit in embryonic form).

You could even do all your research out in the open so peers can identify issues early on.

Why wouldn't this work?!

To give a bit of background to this blog post:

A few days ago, Fernando Perez (creator of IPython, physicist and general science hero) tweeted a few fascinating quotes from an article in National Geographic:

1:41 PM - 22 Nov 13:

"We’re here sharing science. Science isn’t the answers, science is the process." Open science FTW @johnhawks http://newswatch.nationalgeographic.com/2013/11/20/rising-star-hominid-what-we-know-and-dont-know/

1:56 PM - 22 Nov 13 and 1:58 PM - 22 Nov 13

"The most important implication of open access is the change in the scientific culture...... When you have a culture of secrecy, you breed people who trade in secrets."

The article in National Geographic that Perez quotes from is about a high profile piece of academic paleoanthropology where the researchers are doing all their research "out in the open". They fully plan to publish "normal" papers too. But they're blogging about all the cool stuff they dig up, and putting photos of their discoveries online (long before they have enough evidence to write a full paper).

Update 10:30 UTC 30th Sept 2013

Wow: the response from twitter to this post has really taken me (pleasantly) by surprise! This is without doubt the nearest I've had to something "going viral".

I tweeted about this blog post at 15:07 on 28th Nov.

Then, last night (29th Nov) at 20:31, OpenScience retweeted my tweet to 18,534 followers and that started a whole stream of RTs (for which I'm very grateful!). At the time of writing, my original tweet has been retweeted 17 times and a "manual RT" from Tom Hartley has been retweeted a further 6 times. There has also been some discussion of this blog post on those two twitter conversations.

I am seriously thinking that I should ask my supervisor if I can run the remainder of my PhD as an experiment in open science and put all my notes, code, results and data online as soon as I generate them ;)

Comments

when you do paleoanthropology chances that somebody steals your research (or digging site) are pretty slim - when you share some research ideas in (for example) psychology, there are hundreds of better funded researches who can beat you to it in realization (and claim they never heard about your ideas), so I can understand those concerns.
however, I agree with you completely - it is ineffective and we should change it... Open Science Framework, where you can register you ideas, hypotheses and methods and then share them with others could be that "impartial website" - adding forum-like functionality shouldn't be much of a problem

Hi Marek,

Thanks loads for your comment. As you say, some fields will be more susceptible to "intellectual robbery" than others and I hadn't fully appreciated that paleoanthropology is relatively "secure".

On the topic of "intellectual robbery".. part of me thinks that the ideal situation is one where we don't care about "intellectual robbery"; i.e. we do "egoless science" where we don't care who does the research, as long as someone does it so that the community as a whole moves forwards. (This idea is stolen from the idea of "egoless programming" ). However, I suspect this will only work if all scientists are either lobotomised or replaced by robots! Humans naturally want to compete to be the best (and that is often a good thing).

Also, thanks loads for letting us know about the Open Science Framework!

Interesting idea. Getting feedback could certainly be valuable. A lot of projects have flaws that can be spotted even in a brief presentation of the idea. So that's great.

But if it becomes considered bad form to pursue any project that someone else has logged already, then one could have important topics locked up by people with no means (or perhaps even no intention) of pursuing them seriously. If some unskilled grad student declares he is going to work on X, and that means the rest of the world shuns the topic, this would have a very deleterious impact.

So you could say, OK, well let's not have any custom that if someone logs idea X, others leave it alone. But then you see where this leads: then people can get their ideas by perusing the idea board!

Hi... yeah, that's a great point that you might end up with a situation where ideas get "blocked". You could end up with a situation in academia vaguely similar to the situation we currently have with patent trolls ;)

So, I agree: it would clearly be unworkable to have a system where people were forbidden from working on any idea listed on the site by another user. And, likewise, it would be unworkable if all ideas were fair game.

But there are solutions which lie somewhere between those two extremes. For example, if you submit an idea then you must also provide an email address. If someone comes along later, searches the site and finds your idea, then that new person should be able to email you to say "hey, did you ever do anything with idea X? If not, why not? Did you find it was a rubbish idea? If you still think it's a good idea but you just got busy with other stuff then perhaps we could collaborate; all you'd have to do is bring me up to speed on what you already have and you can be a co-author?"

Good discussion developing here. A couple of additional points...

To some extent the peer review of funding decisions aims to achieve at least some of the points Jack laid out in the original blog.

Second. The essence of science is empirical in contrast to just discussing it. For a brilliant dramatization of that see or read Bertolt Brecht's Life of Galileo. The relevant scene is where philosophers discuss whether the moon goes round the earth while Galileo just looks through the telescope and interprets what he observes. Yes discussion beforehand is good - but maybe not too much before getting on with doing the experiment.

Hi Ian,

Thanks loads for the comments.

I totally agree with both of your points. I certainly wouldn't want to have to spend ages discussing each new idea before any experiments were done. And, of course, a fair chunk of discoveries are made entirely by accident.

But my hope is that some ideas could be either improved or rejected with relatively little discussion prior to doing the actual empirical study.

Also, just to mention... several people (on twitter and offline) have made the point that intellectual robbery is already unpleasantly common in academia. i.e. there are people who are willing to steal ideas just to get ahead.

This is very disappointing. I suppose one answer is: can we tolerate a certain "robbery" rate? Say 10% of our ideas are lost through robbery. Maybe it's still worth sharing all our ideas early if the remaining 90% are significantly improved via the process of sharing; such that the benefits of early sharing outweigh the risk of having 10% of our ideas stolen?

I think people's reluctance in this area stems from the fact that they do not want to compromise the perceived originality of their ideas in the eyes of Google-equipped referees, or in terms of the traditional style of call for papers that submissions should be previously unpublished. Also everyone who has done a significant amount of reviewing knows the world abounds with people with absolutely no scruples whatsoever, and would do almost anything, no matter how unethical, to get ahead in whatever way they can - consider the numerous journal papers derived from MSc and PhD theses that others put on the web; the authors of the journal paper change the context slightly and present the work as if it were their own without referencing the source (and many get away with it!), or the way original content authored for a website pops up on the web pages of many other businesses in the same sector as if they wrote it! The vast majority of the people out there are consumers happy to take ideas and software but not that happy to give back (open source software project of mine has had 70,000+ downloads but how many have external people have contributed to the code base? Precious few actually - about 3!)

As for "When you have a culture of secrecy, you breed people who trade in secrets" this strikes me as a bit nonsensical. The whole modern world runs by trading in secrets - people's employability is based on the "secret" knowledge and skills they have individually acquired, our existence as a university is founded on our ability to educate people in the "secrets" of the sciences and every successful private enterprise that I can think of has a "secret" sauce that makes them successful and they don't want to share with others; indeed if they did, they would cease to retain their competitive advantage. That's not to say you can't share and be open about certain things, and this includes ideas (and indeed it can be extremely shrewd, insightful and good marketing to do so) but it does require some thought as to when to do it and how far to take it. Take it too far and you can end up doing the moral equivalent of playing in a poker tournament with your every hand exposed - while everyone else keeps theirs hidden!

"But if it becomes considered bad form to pursue any project that someone else has logged already, then one could have important topics locked up by people with no means ... "

This is true only if you have a problem citing the source of the idea, even if it comes from a unskilled grad student.

Indeed, many (old?) people from the academia seem to think that it is not worthy to give credit to ideas expressed otherwise than by legacy publishing.

Cite and acknowledge the unskilled grad student, then do better, where is the problem?

I had a similar conservation the other day. We started talking about research journals and how old fashioned they are - many people never get their research published at all. This means that there are many people researching and discovering the same things. The Internet was sort of developed to help researchers communicate better, but much of what is published is still controlled by a handful of journals.

I have also heard stories of how the same people make the same discover at around the same time, but it takes years before anybody realises, usually when somebody else does a review of the research and looks at journals in different languages or just from less popular sources.

I think all universities should publish all the research that they are involved with direct to their websites and also publish briefs of what they are working on - not enough to give away too much, but enough to potentially prompt collaboration and advancement.