publication

The growing divide between higher and low impact scientific journals

Ten years ago the Public Library of Science started one big lower impact and a series of smaller higher impact journals. Over the years these publication outlets diverged. The growing divide between standard and top journals might mirror wider trends in scholarly publishing.

There are roughly two kinds of journals in the Public Library of Science (PLoS): low impact (IF = 3.06) and higher impact (3.9 < IF < 13.59) journals. There is only one low impact journal, PLoS ONE, which is bigger in terms of output than all the other journals in PLoS combined. Its editorial policy is fundamentally different to the higher impact journals in that it does not require novelty or ‘strong results’. All it requires is methodological soundness.

Comparing PLoS ONE to the other PLoS journals then offers the opportunity to plot the growing divide between ‘high impact’ and ‘standard’ research papers. I will follow the hypothesis that more and more information is required for a publication (Vale, 2015). More information could be mirrored in three values: the number of references, authors, or pages.

And indeed, the higher impact PLoS journal articles have longer and longer reference sections, a rise of 24% from 46 to 57 over the last ten years (Pearson r = .11, Spearman rho = .11), see also my previous blog post for a similar pattern in another high impact journal outside of PLoS.

plos-not-one_more-and-more-references-over-time

The lower impact PLoS ONE journal articles, on the other hand, practically did not change in the same period (Pearson r = .01, Spearman rho = -.00).

plos-one_same-references-over-time

The diverging pattern between higher and low impact journals can also be observed with the number of authors per article. While in 2006 the average article in a higher impact PLoS journal was authored by 4.7 people, the average article in 2016 was written by 7.8 authors, a steep rise of 68% (Pearson r = .12, Spearman rho = .19).

plos-not-one_more-and-more-authors-over-time

And again, the low impact PLoS ONE articles do not exhibit the same change, remaining more or less unchanged (Pearson r = .01, Spearman rho = .02).

plos-one_same-author-count-over-time

Finally, the number of pages per article tells the same story of runaway information density in higher impact journals and little to no change in PLoS ONE. Limiting myself to articles published until late november 2014(when lay-out changes complicate the comparison), the average higher impact journal article grew substantially in higher impact journals (Pearson r = .16, Spearman rho = .13) but not in PLoS ONE (Pearson r = .03, Spearman rho = .02).

plos-not-one-is-getting-longer

plos-one-is-not-getting-longer

So, overall, it is true that more and more information is required for a publication in a high impact journal. No similar rise in information density is seen in PLoS ONE. The publication landscape has changed. More effort is now needed for a high impact publication compared to ten years ago.

Wanna explore the data set yourself? I made a web-app which you can use in RStudio or in your web browser. Have fun with it and tell me what you find.

— — —
Vale, R.D. (2015). Accelerating scientific publication in biology Proceedings of the National Academy of Sciences, 112, 13439-13446 DOI: 10.1101/022368

Advertisements

The slowing down of the biggest scientific journal

PLoS ONE started 11 years ago to disruptively change scholarly publishing. By now it is the biggest scientific journal out there. Why has it become so slow?

Many things changed at PLoS ONE over the years, reflecting general trends in how researchers publish their work. For one thing, PLoS ONE grew enourmously. After publishing only 137 articles in its first year, the number of articles published per year peaked in 2013 at 31,522.

plos-one_publication-count-per-year

However, as shown in the figure above, since then they have declined by nearly a third. In 2016 only 21,655 articles were published in PLoS ONE. The decline could be due to a novel open data policy implemented in March 2014, a slight increase in the cost to publish in October 2015, or a generally more crowded market place for open access mega journals like PloS ONE (Wakeling et al., 2016).

However, it might also be that authors are becoming annoyed with PLoS ONE for getting slower. In its first year, it took 95 days on average to get an article from submission to publication in PLoS ONE. In 2016 it took a full 172 days. This renders PLoS ONE no longer the fastest journal published by PLoS, a title it held for nine years.

plos-one-rev-time-versus-other-journals-rev-time

The graph below shows the developemtn of PLoS ONE in more detail by plotting each article’s review and publication speed against its publication date, i.e. each blue dot represents one of the 159,000 PLoS ONE articles.

plos-one-is-getting-slower

What can explain the increasingly poor publication speed of PLoS ONE? Most people might think it is the sheer volume of manuscripts the journal has to process. Processing more articles might simply slow a journal down. However, this slow down continued until  2015, i.e. beyond the peak in publication output in 2013. Below, I show a more thorough analysis which reiterates this point. The plot shows each article in PLoS ONE in terms of its time from submission to publication and the number of articles published around the same time (30 days before and after). There is a link, for sure (Pearson r = .13, Spearman rho = .15), but it is much weaker than I would have thought.

plos-one_output-competition-versus-speed

Moreover, when controlling for publication date via a partial correlation, the pattern above becomes much weaker (partial Pearson r = .05, partial Spearman rho = .11). This suggests that much of PLoS ONE’s slow down is simply due to the passage of time. Perhaps, during this time scientific articles changed, requiring a longer time to evaluate whether they are suitable for the journal.

For example, it might be that articles these days include more information which takes longer to be assessed by scientific peers. More information could be mirrored in three values: the number of authors (information contributors), the reference count (information links), the page count (space for information). However, the number of authors per article has not changed over the years (Pearson r = .01, Spearman rho = .02). Similarly, there is no increase in the length of the reference sections over the years (r = .01; rho = -.00). Finally, while articles have indeed become longer in terms of page count (see graph below), the change is probably just due to a new lay-out in January 2015.

plos-one-is-getting-longer

Perhaps, it takes longer to go through peer-review at PLoS ONE these days because modern articles are increasingly complex and interdisciplinary. A very small but reliable correlation between subject categories per article and publication date supports this possibility somewhat, see below. It is possible that PLoS ONE simply finds it increasingly difficult to look for the right experts to assess the scientific validity of an article because articles have become more difficult to pin down in terms of the expertise they require.

plos-one-is-getting-a-tiny-bit-more-interdisciplinary

Having celebrated its 10 year anniversary, PLoS ONE can be proud to have revolutionized scholarly publishing. However, whether PLoS ONE itself will survive in the new publishing environment it helped to create remains to be seen. The slowing down of its publication process is certainly a sign that PLoS ONE needs to up its game in order to remain competitive.

Wanna explore the data set yourself? I made a web-app which you can use in RStudio or in your web browser. Have fun with it and tell me what you find.

— — —
Wakeling S, Willett P, Creaser C, Fry J, Pinfield S, & Spezi V (2016). Open-Access Mega-Journals: A Bibliometric Profile. PloS one, 11 (11) PMID: 27861511

How Long Should a Scientific Publication be?

In one word: short. In two words: it depends.

A neuroscience expert faces the challenge of 100 new neuroscience articles being published on a daily basis. S/he will never be able to read all that. So, what can be done to get your own publication known to the community?

.
1) Know the reader and his/her lack of time
Science reading has become shorter as more and more articles get read per year but less time is spent per article. Within the last 30 years the articles read nearly doubled whereas the time spent per article nearly halved. Renear and colleagues (2009) call this a trend towards ‘literature surfing’ at the expense of careful reading.
Mind that this change in reading behaviour is not enough to compensate for the increase in scientific output. Over the same time span, while the number of read articles nearly doubled, the number of new science publications per year more than doubled. In a new and edgy field like neuroscience this trend is even more pronounced. Here, the output nearly quadrupled (see my earlier post).
scientis; reading; time; efficiency, time pressure
Scientists read more (orange) in less time (blue). This is efficient but is it good?
.
2) Follow publication trends
Perhaps as a result of the reduction in time spent reading each article, scientific publications have become shorter. To give an example, below I plot all reviews and review-like articles published in a well known neuroscience journal (Nature Reviews Neuroscience, as of April 2012). As can be seen easily, there is a significant trend to keep the reference sections shorter and shorter.
references in Nature Reviews Neuroscience
References in Nature Reviews Neuroscience: less and less and less
.
[Update 23/12/2016: this figure has the chronological order of reviews backwards. Reviews in NRN are actually getting longer and longer.]
3) Decide between efficiency and effectiveness
Efficiency: how much more do I get out of a long cumbersome article compared to a short one?
In terms of citations per page, the answer is nothing. Stanek posted a semi-humorous paper on arXiv reporting citations per publication page in the field of Astronomy. Between six and 50 pages there is not much influence of page count. One may call these papers normal articles. However, anything longer will reduce the citations added with each additional page. Curiously, anything shorter will actually increase it. The most efficient paper is 4 pages long and gathers around 16 citations, i.e. 4 citations per page.
Haslam (2010) did a similar analysis in Psychology. He compared short report formats with longer article formats and found reports to usually have a higher per page citation count than their longer cousins.
Effectiveness: how do I maximize the citations my publication can get?
Go for long articles. Stanek found articles around 50 pages long to receive the most citations. Haslam found longer article formats to have significantly higher mean citation counts.
If you strive for efficiency, go for a short report. They receive more impact per page. If you strive for effect, go for a long article. They receive more impact per publication.
.
4) A cautionary note: consider scientific progress
While shorter articles do allow for the faster dissemination of interesting findings, they offer less space to include replications of experimental effects and this can lead to more false positives making it into the field.
Furthermore, the field as such, i.e. some sort of accumulated understanding of what is known in different areas, can disintegrate if articles are not cross-linked through references. Ledgerwood & Sherman (2012) warn of an increased risk to rediscover what we already know because of a trend towards bite-size publications: science as a repetitive rather than cumulative process.
.
In a career sense the most worthwhile publications are the short ones. At least they will get read and cited efficiently. However, later in your career – if you haven’t succumbed to cynicism – you may actually care about science or be hopeful to make it in an academic career requiring a few highly cited articles. In this phase longer, more integrative articles are probably worth it.
Like with so many other phenomena, fast career-minded science is not necessarily good science.

———————————————————————————————————————————————–

Haslam, N. (2010). Bite-Size Science: Relative Impact of Short Article Formats Perspectives on Psychological Science, 5 (3), 263-264 DOI: 10.1177/1745691610369466

Ledgerwood, A., & Sherman, J. (2012). Short, Sweet, and Problematic? The Rise of the Short Report in Psychological Science Perspectives on Psychological Science, 7 (1), 60-66 DOI: 10.1177/1745691611427304

Renear AH, & Palmer CL (2009). Strategic reading, ontologies, and the future of scientific publishing. Science (New York, N.Y.), 325 (5942), 828-32 PMID: 19679805

Krzysztof Zbigniew Stanek (2008). How long should an astronomical paper be to increase its Impact? arXiv arXiv: 0809.0692v1

——————————————————————————————————————————————————-

ResearchBlogging.orgIf you liked this post, you may also like:

Neuroscience is forgetting.

.

.

Figures:

1) from Renear & Palmer, 2009, p. 829

2) self-generated

——————————————————————————————————————————————————-

If you you were not entirely indifferent to this post, please consider to leave a comment.