How to manage theorist's "labbook"?
(Supplementing F'x's answer.)
Physical notebook
I use it all the time, as it is the quickest way to write mathematics and jot diagrams. It's low tech, so it's "always on" (e.g. I cannot run out of batteries).
I have one A4 notebook (so I take it anywhere), with removable pages.
I try to devote a single page only to a single project. On each page I note 3 things:
- date,
- title (or, actually, a pictogram) of the project,
- "page number" for pages on the same project in a given day.
After some time (could be the same day, could be never) I take off pages, to fit them into thematic folders (and usually throwing away most of things, because they are "non-recyclable" rubbish; if it is "recyclable" then I scrap the important things, writing them on a new page).
However, such operation has trade-offs:
- (+) sorted by topic,
- (+) higher valuable content density,
- (+) can be viewed "all at once" by putting pages on a desktop,
- (-) I cannot take all of them anymore,
- (-) it's somehow easier to loose it, unsort, or torn/coffee split/... it.
Electronic notes
Electronic content is much more linkable, searchable, easier to share, more polished etc. However, it takes much more time to write formulas or draw diagrams.
Different things work for different people, but some kind of personal wiki (or a well-organized text file system) seems to work the best.
I'm not settled down, but I'm switching from TiddlyWiki (very neat but requires a browser to run) to Gitit (so I can write in Markdown, with LaTeX inclusions, in any text editor).
I tried Evernote as well (and I'm using it for other content) but for highly-linked content it does not work for me.
See also:
- How do you keep your research notes organized? - MathOverflow
- Is there any efficient non-linear note-taking software?
- Software/App for electronic research notebooks
I cannot say that this is the perfect way to do it, but I'll cover how I do it.
Background on my perspective: I'm a mathematical/theoretical epidemiologist who came up not through applied math or one of the related fields, but through a circuitous route from laboratory biology. So, as far as I can remember, Science Is Done In Lab Notebooks.
A couple work patterns I've developed:
- My lab notebooks are actually notebooks. I experimented with a Wiki for awhile, and just using something like EverNote, but I found they didn't work for me. They didn't capture the same feeling of permanent documentation, and freedom of notation. So now I use notebooks from Black and Red. Mainly because the generic black-and-white composition books remind me too much of undergrad.
- Like wet-lab science, I keep one notebook per topic - but to address your problems, I tend to broaden the topics. A particular focused project (like something I'm being hired to consult for, or a one-year RA, etc.) gets its own notebook. Projects involving the modeling of a particular type of disease all get a notebook. Side projects involving a different aspect of theory...you guessed it, get a notebook. I've also got a small notebook that is literally "random musings and paper ideas". So rather than one lab notebook, I have several. These are identified in ways that are sometimes logical (a GitHub sticker on the very code-heavy project book) and governed by whimsey.
- Always have the book out. Just like a wet-lab notebook, if you're working on a project, you should have the book out. Don't be afraid to write random musings. If you've done some math in your head, write it down. See also diagrams and sketches. Print stuff out and tape it in the notebook, just like one might do pictures of a gel or western blot. In the notebook currently on my desk, I've got several graphs, and even a printout of a question over at CrossValidated that makes up some portion of my thoughts at that point.
- Reference yourself. Occasionally, as you've noted, ideas cross-polinate. I page number all my lab notebooks, so occasionally I will end up referencing myself. "See Lab Notebook X, Page 47."
First, the fact that you're asking the question is already a good step forward. Being a researcher is about managing ideas and data, and some researchers don't fully recognize it!
After a few years as a researcher in theoretical chemistry (all computer, no wet lab), I have seen pretty much every kind of notebook possible:
- Single notebook documenting your progression linearly. I started with that, it hurt a lot.
- No notebook, everything in files: put a 00README file in each topic/subtopic directory, then simply write in there to document your progress. In meetings, either type directly (if you have a mobile device) or write meeting notes then scan them.
- Thematic notebooks (well described in other answers).
What you need is something that works for you. Draw inspiration from the examples above, the other answers, and try it! If at first you don't succeed, change the way you do, identify things that prevent you from fully using the workflow you set out to.
In particular, important factors to be considered are: how much do you travel? where do you work from? For me, thematic notebooks are a no-go because I work from the office, work from home, work from conferences, work from hotels and planes, work from vacation (I know, I know…). Thus, I can't carry around 5 or 10 notebooks.
To be honest, the combination I have seen used very efficiently (by me or people around me) is:
One single “take everywhere” notebook (I make it a Moleskine: because it feels special, it makes me want to write in it). Each lasts about a year. They are organized as follow:
at the back, few pages reserved for key information (important dates we set, strategic research decision, contact numbers for project leads, that very funny quote that had me laughing throughout an entire hour)
at the back, just before that: a few pages per project, with very high-level tracking of the progress (I write about one line or two per week, when the project is running). This allows me to remember
from the front: everything else, chronologically but clearly dated and labeled. Notes during meetings, notes about important stuff running and its analysis when it has finished running, etc.
everything else (long math derivations, paragraphs of text I started to write, full meeting minutes, detailed graphs): electronically
You can further refine: a colleague of mine uses small colored sticky index-tabs on the top of the notebook to index the content that turns out to be the most important (important results on each project, color-coded). I cannot force myself to do that, but I must admit that it really helps her access older information quickly.