Keeping up

[liv]

Some time ago, cjwatson and siderea asked:

it generally seems that working scientists have to spend a sizeable proportion of their time keeping up with other people's research, since after all that's at least theoretically the point of publishing in the first place and unless you're a genius in a tiny field you'll get further that way than by ignoring everyone and striking out on your own! But scientific publications are generally pretty information-dense and there are a lot of people publishing in most fields, so I'm guessing that just keeping up with your reading could use up all your time if you let it. What strategies do people use for selecting out the most important things and keeping the firehose of incoming information under control?

It's a good question, so let me give it a go, albeit belatedly. I'm not sure I can talk about what strategies people in general use, only what I do, but I don't think I'm that much of an outlier.

So yes, keeping up with other people's research is exactly the point. Lone geniuses basically don't exist any more, you have cranks and you have actual scientists who pay attention to others working in the same field.

Point the first, I read fast. That's part of how I got access to academic science in the first place. I am not as fast as a skilled speed-reader, but I can get information out of written text faster than most people I know, and really my acquaintance is biased towards intellectuals and academics. This is also the secret of how I keep up with ~250 journals in my subscription lists across DW and LJ, and yes, I do read every post and don't filter my reading page.

On top of that I did an undergraduate degree which had its problems but one of the things it was very good for was giving me a lot of practice at reading, absorbing and summarizing scientific articles. Basically I had four years of writing three tutorial essays every two weeks, each covering a reasonable bibliography of the key articles relevant to the title. If I wanted to have a social life at all, I had to get pretty damn fast at doing that.

It's not just experience of reading fast, though. Although cjwatson described research articles as generally pretty information-dense, and that's true, in practice there's a lot of overlap in the information. If I'm reading within my own field, I can skim a lot of the introduction section because it's, quite intentionally, a summary or review of recent findings in the field and an explanation of why the research is interesting. Unless something's really directly comparable to my specific research, I won't read the methods in detail, because I'm familiar with the sorts of techniques cell biologists and people who collaborate with us generally use, and I'm pretty unlikely to care whether something was incubated for half an hour or forty-five minutes. And in a good paper the first third of the results should be establishing that the technique is valid, which is important to have but not actually a source of information I need beyond verifying it's there. For example: I deleted this gene, and this experiment shows that when I think I'm deleting the gene, it's actually deleted. Good stuff, and I'd worry if it were absent, but I don't in fact need to slog through it. So if someone with a general academic background, but knowing nothing about tumour suppressors, were to read a recent paper relevant to my research, it might well take them several hours to understand it properly. For me, a typical paper has about half an hour worth of actually new to me stuff.

But even with that, yes, it's a firehose. One of my pet tumour suppressors, the p53 pictured in my icon, is notoriously the cancer factor with the biggest share of the literature, and I often like to quote the fun fact that there's been 80 thousand papers mentioning it in my lifetime, the protein having been discovered the year of my birth. And really, I don't only need to read p53 papers, I need to read about cell proliferation and death, and mechanisms of chemotherapy, and protein synthesis and destruction, and other things that may behave similarly to p53 in whatever respect, and methods that may be relevant to me that haven't been used for p53 yet, and and and.

The answer to this is a combination of automated tools, and peer networks. I tend to lean quite heavily towards the latter. Colleagues working on the same stuff recommend me articles to read, and when I am in a reading phase I do the same for them. And I make sure I go to every scientific seminar I possibly can that might be even remotely relevant to me, because stuff that my colleagues a couple of hops away are interested in enough to invite a speaker is exactly the kind of region to look for articles I'd need to know about but wouldn't find by constructing searches based on what I already know. This worked better when I worked at big famous places like Oxford and the Karolinska Institute (which awards the Physiology and Medicine Nobel prizes, so anyone thinks they're in with a shot really wants to bring their work to the attention of the prize committee), but it's still useful even in a small institution.

Right now we don't have a formal "journal club", a regular meeting where colleagues take it in turns to recommend and give a short presentation on the most exciting article they've read recently, but I do definitely hear about what my PhD students are reading. And some of it is getting up to speed with the foundations of the field, but often when they're doing that they find interesting things which I again wouldn't have thought of. The slight randomness of people following their interests as opposed to automated searches is really good for increased breadth of coverage. And it also means that we're essentially crowd-sourcing the reading time, we don't necessarily all read everything, but people read their own stuff and summarize it for colleagues.

There are quite sophisticated automated tools out there for making sure people notice new stuff that's published in their field, but that doesn't really answer the question of how to go about selecting out the most important things. I have alerts on a smallish number of keywords; with p53 it needs to be in combination with several others, otherwise I couldn't keep up at all, whereas with my #1 PhD student's completely novel cancer regulator, she and I have read every single paper ever published on it, even when they're talking about chlamydia or embryo development instead of cancer. There's a small number of names of researchers I essentially subscribe to, as they're close colleagues in my field and consistently publish good stuff, but even then, I don't read everything they put out. I also have alerts on people citing my work, which is partly a vanity thing, but mainly anyone who thinks my previous work is relevant to them, well, their work conversely is likely to be relevant to me! Of the alerts I get, I screen out about a third of them as being what I think of as workhorse papers; for example if someone's working on one of my drugs, and they try it in a new patient population with a slightly different type of cancer, that's good and valuable work but I don't necessarily need to read it. Whereas if someone proposes the drug has a totally different mechanism from what we previously thought, then I really need to pounce on that pronto.

After that I do a kind of cascading thing. As I read each paper, I'll mark up the bibliography with anything else that is directly relevant. That means not stuff that's ancient, it's really good to cite when a phenomenon was first observed, but if the whole field has accepted it as fact for 10 years, there's usually no point me reading the original paper. And stuff that directly impacts on my own work, for example, if a paper says, p53 causes apoptosis in such-and-such a system, I'm probably not going to follow up citations about the mechanism of apoptosis because I already know that, and I'm not going to follow up detailed descriptions of other features the system has, because I don't work on that system, but I might well need to read the paper that explains what else p53 needs to kill the cells as that could be true in my own systems as well. That usually nets me about half a dozen papers per starting article, except that once I start reading them I'll find that they all cite eachother, so it tends to converge rather than expand.

The other way I limit how much time I spend reading is that I let my reading by guided by my research, rather than by my curiosity. Like, when I was doing my PhD the experimental evidence pointed to the idea that p53 is involved in regulating the cellular machinery for making new proteins. So I went and read a bunch of papers about ribosomes and nucleoli, which would always have been interesting, but it became worth the investment of time once I'd got a novel result showing that p53 has something to do with ribosomes. Or perhaps I see a new phenomenon I haven't encountered before, and I'll go and read about other people who have seen similar things, so I can design experiments to test whether their observations are relevant to my own results or not.

In practice this often means that my reading tends to go in phases; when I've done an experiment which shows something new and surprising, or when I've completed a series of experiments and I'm writing this up for publication, I do in fact spend most of my working hours reading. Oh, and when I'm designing a project and applying for funding, I have to do a fair amount of checking that what I'm doing isn't duplicating effort, as well as justifying why my research question is important. Other times I might only read one or two articles a week; the field moves fast, but not that fast, so if I'm not aware of something new for a few months, it's usually not a disaster. Indeed, sometimes it means I show some conclusion independently, and that can be valuable in itself.

Does that help? Please feel free to ask more questions, including the rest of my readers beyond the ones who asked me in the first place.

Comments

[seekingferret]

Great post. I've always been fond of the image from the beginning of John Brunner's novel Stand on Zanzibar whose job was to sit in a carrel all day reading randomly assigned journal articles and passing on notes to experts in relevant fields if his random reading happened to spark a possible connection between two disparate papers. Brunner called the job 'synthesist', I think.

In my engineering the focus is much more on proven technologies, and I tend to only read journal articles when they've been recommended as important by someone in my peer network- colleague, industry friend, salesperson, etc...

[(Anonymous)]

A fascinating account.

My first job was in an industrial research laboratory concentrating on long-term research. The Director had a rule of thumb that, when he gave a post-doc research chemist a new project, the Director did not expect to see him or her for two weeks because they should be in the library full-time. (Yes; we had plenty of women research chemists even in the late 1960s.)

You will appreciate from the above dates that all the journals were on paper at that time.

Southernwood

[khronos_keeper]

Yeah, agreed. Part of getting an advanced degree in any scientific field is essentially learning how to drink from the firehose without drowning.

And by that I mean the fine art of skimming effectively, and learning how to determine how much of any work, article, or project is relevant to 1.) your current project, and 2.) how you can consider it for future work.

But also your description of work taking phases is also very important, but it's also reiterative and recursive. Many scientists will initially find something promising, determine it might be interesting/useful, do enough cursory reading to elaborate on above, and then find it again later.

I recently remembered that I had a thing for survival analysis like a year ago, and on finding it again figured it would be interesting to see if I could use it in my reseach.

[hairyears]

Is there any truth in the cynical canard that there's an inverse relationship between publication volumes and clinical benefit?

As far as I'm aware, Gendicine's the only treatment out there, making use of detailed knowledge of P53.

That's not to say that an advance understanding tumour-regulating genes is unimportant: but it's a hint that there's a 'bandwagon' effect, with the vast majority of papers utterly devoid of interest to anybody making progress in the field.

[silveradept]

And in all of this are the librarians scrambling to try and find ways of presenting the material so that these connections can be made. We can only hope we succeed in some small way at some point.

[ephemera]

*reads with iinterest*

Related query - how do you go about taking / keeping notes of what you've read and which thing was mentioned where so you can find them again? Or does the level of "Oh, rats, I know I read something about p53 and widgets that would be really relevant here" information your memory can hold give you good enough search terms to re-find the article without such a system? (I wonder if that's a possible difference between humanities and sciences, where being able to remember the basic thesis of a paper doesn't reliably mean you can find it again, even if it is electronic, and quite a lot of things still aren't)

[quizcustodiet]

This element of my D.Phil. was a significant contributor to deciding to leave academia! I was perfectly able to read and understand papers in my field, but I didn't have the enthusiasm to do more than the minimum required to get the D.Phil., which is a clear indication that I'd have made a terrible researcher! (For the record, there are many reasons that I would have been a terrible researcher, but this is the one most relevant to your post!)

The colleagues who went on to make successful careers in academia are largely the ones who had the enthusiasm and energy that they'd be in the lab 9-6 Monday to Friday and then doing their journal reading in the evenings and weekends to stay on top of the field. I have mixed feelings about that: on the one hand, I think it's admirable that they have that level of dedication, but on the other hand I think it's unjust that academia is set up in such a way that you need to work excessive hours to make a successful career, especially as the salaries are very low compared to other fields requiring similar levels of time commitment.

[switterbeet]

I am curious about this as well! I have started using Mendeley to highlight/record notes on papers which seem relevant, and sometimes it is ok for re-finding articles if I can remember enough key terms/author names etc. But I was only a MSc student, not quite as steeped in research and reading.

I also wound up using a spreadsheet to track citations as I researched for actual writing later, with columns of "section", "topic", "citation", and "quote/summary". So when it came to writing "Intro, stuff about topic A" I could easily filter to the references I'd already read on that topic.

Very interested in how others go about it!

[liv]

That sounds like an absolutely classic, and non-snarky, example of the difference between scientists and engineers! And I've always loved the idea of a professional synthesist. I think Brunner more or less was one IRL, he was employed as a "futurist" consultant, he got paid for knowing stuff about all kinds of disparate things and making connections.

And I kind of am myself in a small way; it's definitely noticeable professionally that I have a broader definition than many academics of what counts as relevant to my work, and I'm often the one to inform colleagues of potential connections to tangentially related stuff. And even this journal, I read a lot, and fairly eclectically, and my best posts are ones when I bring together ideas from different aspects of what I read.

[liv]

Yes, definitely the start of a project is key reading time. I was just at the transition from paper to electronic; as an undergraduate I had to do all my reading in the library, but by the time I finished my PhD nearly everything was online. But I do prefer to print out journal articles and read them with a pencil in hand to mark them up, partly because of the habits I learned at Oxford.

[liv]

Good examples. I think there really is a skill to skimming and working out what's relevant, as well as keeping things in mind to come back to. I found it interesting to sit down and consider how, practically, I do this, because reading and absorbing a lot of technical information quickly is what I've been doing for the whole second half of my life.

[liv]

No, there is most certainly not an inverse relationship. There are really a lot of publications on receptor tyrosine kinases, and there are hundreds of thousands of people who are alive today specifically because we understand how RTKs work in detail. (Not the best article but the most recent I can find that isn't paywalled.)

p53 itself is a strange one; a lot depends on what you mean by making use of detailed knowledge here. p53 is not a very druggable target, we've always known that, because it's a transcription factor; you can't easily manipulate it with drug-like small molecules. Now that oncolytic viruses are starting to be licensed in Western healthcare systems I wouldn't be surprised if we see a lot more Gendicine-style direct introduction of p53 into tumours, a treatment approach which is widely and successfully used in China now. I mean, you can say "only" Gendicine and "only" China, but there's a lot of people with head and neck cancer in China and the Far East whose lives have been saved because of detailed knowledge of p53.

On the other hand, p53 is involved in so many processes, so there's all kinds of treatments which make use of knowledge of p53 indirectly. Conventional radiotherapy and many forms of chemotherapy are quite p53 dependent, so maybe knowing about p53 doesn't directly lead to new therapies, but it does offer a lot of opportunities for predicting treatment success or failure and therefore informs decisions about treatment. And there's several drugs being used to treat patients whose development relied on knowledge of p53, even if the specific mechanism of those drugs isn't directly or purely about p53 activity.

I mean, you can say all this is utterly devoid of interest if you like; that's pretty much the Impact agenda, isn't it, there is only funding available for research that directly and demonstrably leads to medical progress. I personally think this is a really short-termist way of thinking about science; I don't think any researcher, let alone any government functionary, can really predict which science is going to lead to better cancer drugs in years to come, because we don't even know what the science is going to say until we've done it! And yes, people do jump on bandwagons and publish poor quality papers in trendy areas, but that's at least partly caused by the attitude that it's only worth doing science if you can see how it will lead directly to new drugs.

I heartily recommend Brooke Magnanti's recent survey of drug discovery and why it's broken; indeed I may make a top level post about her piece soon.

[seekingferret]

That sounds like an absolutely classic, and non-snarky, example of the difference between scientists and engineers!

Well, not really. :P I think there's a spectrum of product development and where you stand on the pipeline affects the kind of literature you need to read. There are pure scientists and pure engineers but there's also a lot in the middle, I think. I mean, I build tools for material science researchers, so to some degree (and the degree depends on the tool technology, on intellectual property issues, on the complexity of the underlying physics, and other issues) I need to speak their language.

[liv]

All the love for librarians. Including the ones who are working with digital curation and making tools to help with the connection making.

[liv]

That's a good question too. I generally print out papers, which is not very environmental, but my brain is quite kinaesthetic and the physical act of highlighting helps me to absorb information. And then I file them according to general area, then alphabetically (because I'm more likely, though not certain, to remember the first author's name than the year of publication). And I try to have a reasonably good index of what's in my files. A lot of stuff is findable again by repeating the search, because yes, things in science tend to have more distinctive names than in many fields.

[liv]

I don't use the modern automated tools as much as I should, partly because I started my education without them, and I never quite have time to make learning new tools a priority. I somewhat prefer Zotero over Mendeley, but honestly, I pretty much rely on a giant spreadsheet myself. Also anything I've actually written up in a formal way will have a bibliography, so I'll sometimes rely on my own article drafts, grant applications etc to dig out that paper I remember reading.

[liv]

Whereas for me reading new scientific articles is sometimes procrastination from other parts of my job, I'm basically always excited to read about more new sciencey findings. But I absolutely agree about the way that academic career success depends on being willing to do way more than you're actually paid to do. I did have that enthusiasm when I got to the end of a PhD, but the past ten years have more or less killed it. It's not at all sensible that the only way to succeed is to do the actual interesting science in your own time. I didn't mind doing lab work during the day and reading and writing up in the evenings, but at my level I'm mostly doing admin and urgent but not long-term useful stuff 9-5.

[hairyears]

Thanks for Dr. Magnanti's long read article!

I'm a regular reader of Nick Lowe's In the Pipeline blog, and studied a little pharmacology as an undergraduate, so that's pitched quite well for my level.

We could've done with it on the Pharma analyst's desk in Credit Lyonnais, back in the day. The level of understanding among 'experts' and analysts in the City is at best, somewhat superficial.

[switterbeet]

I started with Mendeley by a fluke and sort of stick with it out of habit. I've seem some really rad ways of creating bibliographies with Zotero though... just not enough to get me to learn a new thing. :P Thank you for sharing again!

[green_knight]

This is utterly marvellous and I love it.

Could you get a cleaned-up version of this posted to your university's website in the next three weeks or so? (Kidding about the timeframe, but I think it would be really, really useful to students who are just completely overwhelmed). The reason I mention 'three weeks' is that I'm currently editing an article where 'academic reading techniques' are covered, and this would be a perfect source to mention if it were available from a university site/non-personal blog.

[green_knight]

I'm pretty much the same, only I wrote a database to track things in my second term so never got around to learning other people's tools. Part of the problem seems to be that in order to profit best (retrieve reference in any conceivable format) you need to spend a lot of time entering data. This means that if you're citing things a lot in different venues it's totally worth the effort, but if you cite something at most once, your faster editing things manually.

[liv]

Thank you for this far more nuanced description! Obviously there's a whole range of things that come under the heading of "engineering", and some of it is going to be more conservative and some more neophile. And definitely being able to talk to scientists is an important part. Do you ever talk about your work in your journal?

[liv]

I think some of the problem that Magnanti is pointing out is precisely that pharmacological research doesn't fit too well into the traditional system of venture capital investments, risks and returns. And yes, part of that is that financial people don't understand the science.

[liv]

Aww, thank you, I'm really flattered! I can't think of any mechanism to get it published anywhere official, but it's really nice you think it could be useful. I feel like this post isn't really advice, as such, it's more a statement of the fact that staying on top of reading comes with experience. But yes, I can imagine an overwhelmed student finding it helpful to hear how a more experienced researcher works.

[liv]

I think ideally these things should be findable by dois and the like, without having to manually add to the system all the metadata by which you might retrieve references. But that's idealistic, and there is definitely a trade-off in how much time it's worth investing into setting up automated tools compared to how often you're going to retrieve things.

[seekingferret]

Not very much, no, for a combination of 'lots of work is things I can't talk about in public for IP reasons' and 'I assume anything I post online could theoretically get back to my boss'. I would like to post more about work- I really like the posts you make and think I could write things about my work that are similarly interesting, but haven't figured out a way to do so effectively given those constraints.

[hairyears]

Drug development has been shoehorned into a 'market' solution where the rewards for success are underwritten by the award of a monopoly by governments - and this market's incentives are not at all aligned with the benefits that governments desire.

Not even when these governments are dominant market players in the purchase of new drugs, and I find that surprising.

Which is to say: hedge funds and other specialists are doing what they do best - arbitrages and 'special-circumstances' trades where markets are distorted or opaque - whenever they invest in Pharma, at any stage of company development.

Meanwhile, there is no market, however artificial, that aligns the cost structures of drug discovery with the outcomes, however priced, let alone with pricing structures which align the profits with with some model of societal benefits.

Which makes it somewhat surprising that the state is not a much, much bigger player in drug discovery, with a carefully-constructed market for drug development.
A 'flow' based model of research - continual gain in knowledge, rather than results defined by profit - is far easier for state-level actors than for commercial entities. So, too, collaboration and the publication of results.

I guess too many people have too much invested in the Patent model, or in pharma shares, and we've all forgotten that this is an artificial market.

The place where I would look for signs that this is ever actually changing would be the US Department of Defence - troops get deployed into disease-ridden disaster zones with dismaying regularity and the DoD are used to interacting with the private sector on a very different model for their R&D.

[untonuggan]

My upper level undergraduate history classes were about source evaluation: finding them, drawing conclusions, evaluating if they were "reliable" or not (or if you could still draw useful conclusions from the biases presented; pro tip all sources are biased), etc.

Now that I have chronic health stuff, it's been *really* useful in evaluating health things. Like, "should I do this treatment instead of that one?" or "who is a doctor for this obscure health condition in my area who might take my insurance and have openings for appointments this year?" or "what new research is being done for my condition?"

Also useful for keeping myself entertained. I just wish my local libraries still offered JSTOR access. :(