Tag Archives: tools

Tools of the Trade: Timestamps and Text Editors

Quoting people accurately is a mundane but crucial task in my daily work. Over the years, I’ve tried a variety of approaches, but I think I’ve finally evolved the most efficient system for handling it. When I’m conducting an interview or attending a presentation at a research conference, I take copious notes. These used to be on paper, using a complicated note-taking technique I developed in college, but after discovering the amazing capabilities of Emacs (the One True Text Editor), I’ve switched to doing it on a computer.

My notes are not strict transcripts. I tried doing transcripts, but it was more hassle than it was worth, and I found that there’s a conflict between transcribing someone’s words accurately and actually understanding what they’re saying. Instead, I note the important parts of the conversation as if I were attending a lecture, and periodically insert a semi-transcribed snippet if they start saying something I think might be quotable. Such a snippet might read “thr’s bn a lotta contrvrsy n th fld abt this, bt itot tht th mech is rly simpl.” With that sort of primitive shorthand, I can transcribe even relatively quick talkers with about 90% accuracy in short bursts. But I want 100% accuracy.

To get that, I record all of my interviews and conference sessions. Yes, recording phone conversations is completely legal everywhere, as long as you ask permission first. The notes plus the recording give me all the information I need, complete with 100% accurate quotes. To streamline the process, though, I added one more piece: timestamps. As I’m taking notes, I hit a keyboard shortcut, and my computer inserts the elapsed time since I started the recording. When I’m writing, I can look through my notes to find the quote I want, then zip to that time in the recording, play back the tape, and check the quote. Cumulatively, this can shave hours of tedious rewind-play-forward-play searching on a long assignment.

Unfortunately, most word processing software doesn’t offer elapsed time stamps, and because the popular choices are either closed-source (Word, Pages) or mind-bogglingly complicated open source (OpenOffice), it’s not trivial to add this feature. You could sort of work around the problem by using real-time stamps, then calculate the elapsed time between when you started the tape and when you got to a particular point, but that could take as long as just searching the recording blindly. Instead, I suggest switching to a proper text editor (a choice that has a lot of other benefits, too). Once you’ve done that, there are at least two ways to add the elapsed time feature.

For Emacs users, the solution – like the solution to every other Emacs need – is to modify your “.emacs” customization file. It’s not hard. Here’s the bit that does the actual work:


;;; Elapsed time and realtime stamp functions
(defun clock-start ()
"Starts a clock which can then be used by
`clock-insert-elapsed' to insert elapsed time into the current buffer."
(interactive)
(setq clock-start (current-time)))

(defun clock-time-elapsed ()
"Returns the elapsed time since the clock was started with `clock-start'."
(interactive)
(setq elapsed-time (time-since clock-start))
(format-time-string "%T" elapsed-time t))

(defun clock-insert-elapsed ()
"Inserts the elapsed time since the clock was started with `clock-start'."
(interactive)
(insert (clock-time-elapsed) " ")
(newline))

(defun timestamp ()
"Insert string for the current date and time."
(interactive)
(insert (current-time-string) " ")
(newline))

To link that to some convenient shortcut keys, also add these lines:


(global-set-key [(hyper f8)] 'timestamp)
(global-set-key [(hyper f7)] 'clock-start)
(global-set-key [f8] 'clock-insert-elapsed)

That’s it. Now, hitting command-f8 inserts a real-time stamp, while command-f7 starts a stopwatch and f8 by itself inserts the elapsed time since the stopwatch started. Whenever I start recording a phone call or conference talk, I simply hit command-f7 right after pressing the “record” button. Whenever the source says someting potentially interesting, f8 then bookmarks it for later access. These keys are convenient on a Mac – if they’re already assigned to something else on your system, simply choose different shortcuts. Once you’ve used them a few times, they’ll become second nature.

For other text editors, I use a pair of Bash scripts that I can call from a general system-wide shortcut. These are incredibly primitive scripts, and I’m sure there’s a more elegant way to do this, but here’s my approach. First, we have a script to start the clock, creatively named starclock.sh:


#!/bin/bash

if test -e ~/Desktop/clock
then
rm ~/Desktop/clock
fi

echo $(date +%s) > ~/Desktop/clock

That saves a file to my desktop with the current time, in seconds since the Unix epoch date (don’t ask – it works). The other script, etime.sh, looks like this:


#!/bin/bash

BEFORE=$(cat ~/Desktop/clock)
AFTER=$(date +%s)

# Calculate elapsed time in seconds and subtract 19 hours (68,400 seconds)

ELAPSEDSECONDS=$(expr $AFTER - $BEFORE - 68400)

echo -e $(date -r $ELAPSEDSECONDS +%H:%M:%S)

That reads the clock start time from the desktop file, then calculates and returns the elapsed time since the clock started. Set a pair of keyboard shortcuts (such as command-f7 and f8) to call these two scripts and insert the result of the latter into your text document, and you’re all set. This has worked well for me in both Vim and the free version of Smultron, but any text editor that can call command-line scripts should be able to handle it.

Yes, I understand how useless this post will be to most of my readers. If you’re a scientist working in the lab, you probably have no need of timestamps or interview recordings. If you’re a journalist, you probably have no clue how to make a text editor execute a script, how to modify your .emacs file, or even whether any of this is safe. However, I hope it provides a good example of why you might want to learn those things. For modern writers, editing text on a screen is a fundamental job skill, and the software we use to do it is an essential tool. If you’re limiting yourself to corporate-oriented word processors, you’re like a photojournalist trying to shoot an assignment on a crappy camera phone. It might be possible, but it won’t be easy. Text editors are the digital SLRs of writing, and learning how to use and modify them is time well spent.

Tools of The Trade: Zotero

If you read a lot of scientific papers, or do any other kind of academic study, you need a way to manage the mess. I’m talking about the piles of photocopies, books, and notes – or these days, gigabytes of PDFs, web links, and text files. Zotero is the best solution I’ve seen for this so far.

The genius of this little browser plug-in is that it automatically senses certain types of web content, such as research journal articles and bibliographic searches. It has a particularly profound understanding of PubMed, the main search engine for biomedical science. Zotero doesn’t flaunt its knowledge, though – it just adds an unobtrusive icon in the address bar when it understands what you’re reading. Click that, and you can add citations to a library the program maintains on your computer. There’s also an option to synchronize those citations across other devices, using Zotero’s server. All of this is free.

Zotero in action.

Zotero in action.

Once the citations are on board, you can get to work. The screenshot shows a paper I was reading today. The top part of the screen is the regular browser window, with a paper I was reading online. The bottom section is the interface for Zotero. With a click, I can make Zotero take up the whole screen, or open in a separate window, or go away entirely. To save clips of text from the paper, all I have to do is highlight some text, right-click to get a contextual menu, and then select “Add to Zotero note.” The clip gets stored alongside the citation for easy reference. Of course I can also add my own notes, and can export the notes, the citation, or both in a wide range of formats. There’s also a slew of collaboration tools that operate through the program’s web site.

As with any sophisticated tool, there’s a bit of a learning curve, but so far I’ve been able to figure out how to do just about everything I’ve wanted to with the application. If I find myself thinking “Zotero should be able to do that,” a few minutes of clicking around usually reveals that it can. There are other citation managers available, of course – Mendeley and CiteULike both have dedicated followings – but for what I do, Zotero has been a good fit.

Tools of The Trade: Stand-Up Desk

This is the first of what might turn into a series of posts about tools and tricks I’ve found handy for my work. I figured I’d start with the thing most people would notice first on walking into my office: my standing-height desk.

I switched to working on my feet in the summer of 2010, so if you’re wondering whether I stole the idea from über-geek Gina Trapani, I didn’t. While I was ahead of that high-profile example, I’m hardly a pioneer.

In my case, the desire to stand grew out of a simple need for a new chair. While wasting way too much time shopping for one, I stumbled on the stand-up desk idea, and liked it immediately. Rather than buy a new chair, I could simply adjust my desk and dispense with the seat entirely.

Like Gina, I’m fortunate enough to own an Ikea “Jerker” desk, a brilliantly designed piece of furniture that the company has since discontinued. An afternoon’s work raised everything to the appropriate level. Most of the time involved unplugging, removing, then reinstalling my computer and its connected peripherals. I also borrowed a tall stool that was sitting around the house, so I could take occasional breaks from standing while building up my “work legs.”

Standing height desk.

This is where I take a stand.

After the first two weeks, my endurance was up to about four hours. Within a few months, I could stand for most of the day without really noticing. It’s just the way I work. I wasn’t doing this for the exercise, but gained some leg strength nonetheless; my comfortable bicycling pace increased a gear or two. I’ve also lost the pain that used to creep into my spine after a long day of sitting, and my computer is above the easy reach of a six-year-old child, a feature that’s quite handy in my house.

The transition wasn’t painless. I needed the stool a lot at the beginning, and it’s still useful from time to time. I also discovered early on that long periods of standing demand comfortable footwear. My office is uncarpeted, so anything less padded than tennis shoes gets tiring pretty fast. The exact height of the desk also seems to matter more now than it did when I was sitting. Belt height is perfect for working at the keyboard. Writing by hand, however, calls for a surface closer to the bottom of my ribcage, so on the rare occasions when I do that I sit on the stool and lower it a bit.

Overall, I’m quite pleased with this arrangement. I’m not planning to invade Iraq anytime soon, but for the less complex – and generally more successful – projects I do, a high desk works quite well. It seems to make me focus more on what I’m doing and find fewer distractions. In other words, it makes me a more upstanding worker.

Cell Is for Cellulose

Cultured cells are a mainstay of modern biomedical research, but they’re very limiting. Growing the cells in a flat monolayer on a petri dish is the easiest way to control and study them, but it doesn’t represent their normal in vivo environment. Culturing them in a 3-dimensional matrix encounters the other horn of the dilemma: the cells are growing in a structure more like a real tissue, but they’re much harder to maintain.

Now, the prolific lab of George Whitesides has hit on an astonishingly simple and cheap solution:

Ratmir Derda, a postdoctoral student co-mentored by Whitesides and [Don] Ingber at Harvard’s new Wyss Institute for Biologically Inspired Engineering, has realized that by growing cells on several sheets of uncoated paper, he can solve a problem that has bedeviled biologists for years: how to easily grow and study cells that mimic the three-dimensionality of real tissue.

3D image of epithelial cells growing on cellulose. Image courtesy Ratmir Derda.

3D image of epithelial cells growing on cellulose. Image courtesy Ratmir Derda.

You read that right: paper. They’re taking plain sheets of blotter paper, sterilizing them, and dipping them in a cell-impregnated nutrient gel. They can then fold or layer the paper to create whatever 3-dimensional structure they like, all while maintaining the accessibility and reliability of flat cultures.

The work supposedly appeared in the 19 October issue of PNAS, but I can’t find it on the journal’s web site. Hopefully they’ll put it online soon, because I’m sure there are a whole lot of cell biologists waiting to try this out in their own labs.

Stick It In Your Ear, Then Slap Your Forehead

Ever see some new device, and say “wow, why didn’t anyone think of that before?” I mean, really say it and mean it – not sarcastically, and not as part of a hackneyed marketing phrase, but because you really, truly wonder how this invention escaped discovery for so long. That doesn’t happen very often, because let’s face it, most technological development is pretty easy to see coming. The other day, though, I bought one of these:

Olympus TP-7 microhpone

It’s a recording microphone for telephone conversations. Looks like an ear bud, doesn’t it? That’s because, in contrast to every other ugly hack I’ve tried for this application, it doesn’t plug into the phone wire or suction cup onto the phone receiver. No, it plugs into your ear, quietly interposing itself between the phone receiver (which echoes both sides of the conversation already) and your ear. Get it?

Besides the jaw-droppingly obvious (in retrospect) design concept, it also has nice construction and packaging, with three different sizes of earplugs, a set of adapters for different recording devices, and a classy-looking brushed silvery exterior – and the sound quality is excellent. If you need to record phone calls, you need this device.

Wikipedia for Proteins

Genome Biology just published an interesting paper by a group of prominent protein chemists, plus Jimmy Wales, the creator of Wikipedia. In a nutshell, these folks have developed a new wiki-based protein annotation system, which seems like a wise way to approach the enormous problem of organizing information about these molecules. In an accompanying press release, they explain the work this way:


The source material for WikiProteins comes from a mixture of existing authoritative databases (such as the Unified Medical Language System, UniProtKB/Swiss-Prot, IntAct and GO), supplemented by concepts mined from scientific papers published in public literature databases. The automated data mining identifies ‘facts’ in these available resources, such as protein functions or protein-disease relationships. This process created over one million biomedical concept clouds – called ‘Knowlets’ – around each individual concept. The developers of the site now hope that many researchers will follow their call to annotate, via WikiProteins, the Knowlets for which they are leading experts. The method enables researchers to add data even from sources that are not openly available, such as from journals only accessible via publishers’ databases, immensely enhancing the potential for comprehensive coverage. Each page of text called up via the system is automatically indexed and concepts are connected to the WikiSpace, so that their definition comes up and the information can be edited directly from the page.

I wish they could have accomplished it without feeling the need to coin a new buzzword (Knowlets? Didn’t they hang out with the Hobbits?), but hopefully the system will evolve into a useful tool as users edit it. If you’re anxious to test drive it, there’s a sample page up here.

Crossing Over

I’ve been struggling with a tricky problem in Web design lately: how to preview my sites in Internet Explorer. For my personal site, I largely ignore that buggy browser’s bizarre behaviors (see my archived tirade for details). But lately, I’ve been working on a Web redesign for a nonprofit group, and they need their site to be accessible. There’s no current version of Explorer for the Mac, and my old copy of IE5 doesn’t capture a lot of the more recent bugs the folks in Redmond have added. It seemed like the only reasonable option was to install Windows on my Intel Mac, paying hundreds of dollars to Microsoft to get a copy of their operating system, and then install Explorer. I have a moral objection to that, though, as it would mean enriching Microsoft, not in spite of, but because of their very worst habits.

Screenshot of Explorer running inside Crossover on the Mac.

Thankfully, the good folks at Codeweavers have come to my rescue. Instead of sending a big payment to Bill Gates, I can now send a measly $60 to this small company that helps support the Open Source movement. That’s the registration price for Crossover, a product very similar to the more popular Parallels. While both applications allow one to run Windows applications on the Mac, with Parallels you still have to shell out for a copy of Windows itself. Crossover eliminates that by using the open source Wine emulator instead. Now, back to hacking these stylesheets.

The Joy of Text

Over the past several months, I did one of the hardest things for a professional writer: I switched to a new word-processing application. This is the third time in my ten-year writing career that I’ve done this. Shortly after leaving the lab, I migrated my work from a “borrowed” copy of Microsoft Word (The Worst Word Processor Ever Created) to the vastly underrated AppleWorks application. When Apple abandoned that product, I moved to the supremely elegant Nisus Writer. I’ve heard that for some writers, this application was their main reason for buying a Mac. While I have other excuses, I can understand that thinking perfectly. From its uncluttered, easily configurable menus and tool drawer to its open, nonproprietary file format, Nisus is truly a writer’s tool.

If I were not such an extreme geek, I would have stuck with that decision, and I still recommend Nisus Writer to aspiring and practicing journalists who aren’t especially technical. Besides its aesthetic advantages, having all of your notes and past stories in text-based files (specifically Rich Text Format, or RTF) ensures that your archive will remain accessible for years to come. You can’t say that about many other formats.

About a year ago, though, I started to re-learn computer programming after a 20-year hiatus, and it changed my perspective on “word processing” completely. You wouldn’t know it to read their emails, but in their regular work, computer programmers are the most diligent copyeditors on the planet. As a result, they’ve developed text-handling systems that make even an elegant word processor like Nisus Writer look like a dull #2 pencil. If you’re a technically-minded writer, especially if you work for “new media” outlets on the Web, keep reading.

A modern application, such as a Web browser, consists of two types of files: source and compiled. The compiled program is what most of us want; it’s the set of machine-readable instructions that tell the computer what to do. Indeed, most commercial software is only available in compiled form. But programmers don’t write compiled code. Instead, they work with source code, a series of text files containing human-readable, or at least programmer-readable, versions of the instructions that will get turned into the final compiled code.

The source code for an application can easily run to hundreds of thousands of lines of text, and it’s typically spread across dozens of separate files. Programmers have to be able to keep all of that text organized, search through it in myriad ways, edit it, and coordinate their work with their colleagues. It’s like writing and editing a very large collaborative book, except that books are much more tolerant of errors – a single typo in the source code of a program, even a missing semicolon, can derail the entire application. To handle this mammoth task, they turn to text editors. The best programs in this class are all free, and they’re worth a look for anyone who deals with complex texts.

In the photo below, for example, you see a screen capture from a recent project I worked on for the New York Academy of Sciences. Inside the window, I’m working on two sections of the document simultaneously in a split-screen view. The top part contains the text I’m writing, and the bottom contains my notes from a seminar on the topic. The file is stored as plain text, so I can add or edit HTML tags directly and even “publish” it on my local hard drive as a Web-browser-readable document. Plain text files are also the most durable archival format available for digital media; we can read text files stored three decades ago, and we’ll be able to read plain text files three decades hence.

Split-screen view of Emacs using Org-mode to write a story..

The application pictured is one of the oldest computer programs still in regular use, a text editor called Emacs. While its history is long (I’m using Version 22.1.1), Emacs is certainly not outdated. With constant revisions by a large and vibrant open source community, it is still the standard text editor for many programmers. It also has a very small but ferociously devoted following among professional writers, some of whom sing its praises quite loudly.

Emacs doesn’t have a learning curve. It’s more like a cliff. If you install Emacs (it’s available for all modern operating systems) and simply fire it up, hoping to learn as you go, you will get absolutely nowhere. That’s because Emacs predates graphical user interfaces. While it uses modern windows and menus now, its command-line heritage is still visible everywhere. The keyboard “shortcuts” it uses also predate modern standards. For example, to quit Emacs, you use the sequence “ctrl-X ctrl-C”. That made perfect sense back in 1975. It would still make perfect sense, if that young whippersnapper Steve Jobs hadn’t come along and decreed “command-Q” as the standard shortcut to “quit.”

Jobs also spoiled everyone into believing we didn’t have to read any manuals. Emacs, in that respect, is implacably old-school. You can read the manual completely, then install and use Emacs without difficulty. Or you can prod around in the dark hoping to figure it out on your own, suffer hours of agonizing frustration, pore over error messages that seem deliberately obtuse, and finally, sheepishly, read the manual. One way or another, you will read the manual, or you will fail.

For those with the patience to learn it (I took about a week to become competent with it, and a couple of months to become moderately proficient), Emacs yields tremendous rewards. The keyboard shortcuts, while initially cryptic, eventually reveal their own logic. They’re also numerous. Though the mouse is often handy in Emacs, it is never really necessary; the entire application is controllable with just the keyboard. That’s a surprisingly useful feature when transcribing a fast-talking interviewee on a laptop. Because of its programming heritage, Emacs also offers a vast array of sophisticated text manipulation options, going far beyond the simple search/replace features of a standard word processor. There’s also an active community of Emacs hackers providing all manner of plug-in “macros” for customizing it, and the programming language underlying the application (Emacs Lisp) is simple enough that even novices can modify it themselves. Within days of installing Emacs, for example, I had added a line-wrapping feature and a word count algorithm.

Ironically, I never use Emacs for my programming. Every Mac comes with a suite of professional programming tools, including the Xcode development environment and text editor. Xcode has nearly all of Emacs’s advantages, but it adds a truly useful graphical interface. The only reason I don’t use Xcode for my regular work is that it’s too hard to customize, and there are a few missing features (such as word counting) that I simply can’t live without.

Smultron was also in the running as my editor of choice for awhile. With built-in word counting, elegant tabbed file organization, and one of the slickest, easiest-to-use interfaces of any application I’ve seen, it’s the Nisus Writer of text editors. Even a nontechnical writer could learn Smultron in a few minutes, and would quickly discover that it’s a far more agile composition tool than any business-oriented word processor. Among its many other endearing traits, Smultron is tiny, so it opens very, very fast. It can take half a minute for Word to lumber onto the screen from a cold start, even on a fast system. When I click the Smultron icon, I can start typing less than two seconds later. No more groping around the desk for a pen to take notes when some long-sought source suddenly calls back.

So why did I pick Emacs? I had all but settled on Smultron for my writing when I stumbled on an obscure little plug-in for Emacs, and it revolutionized the way I work. Look again at the screenshot above, and notice the orange and purple highlighted headlines. Those come from Org-mode, a note-taking/organizing/outlining system so astonishingly useful, it will be the topic of a separate post.

Just Say "No" to Microsoft Word

Microsoft Word, one of the worst pieces of software ever to enter widespread use, has been a thorn in my side for years. As a writer, it offends my aesthetics. As a techie, it offends my sense of good design. And when it comes to digital security, it just plain offends. There are plenty of other tirades against Word on the Web (such as this, this, and this), and reams of information showing how this application seriously hinders regulatory compliance and technical development for all sorts of organizations. When Microsoft finally lost the ISO standards war on this issue, many people hoped that Word’s corrosive influence on the world would soon come to an end.

No such luck.

With the release of Vista and the newest version of Office, the demons of Redmond have unleashed another round of document exchange agony. It’s called .docx. The .docx format was Microsoft’s bludgeon in the ISO standards war they just lost (the free, powerful, open source alternative Open Office won). Besides being a dead-end standard now, .docx suffers from a couple of other fatal flaws. First, it’s completely incompatible with prior versions of Word, so if you use the new Word and send a file to someone using the 2004 version, they can’t open it. Second, there’s no Word version for the Mac that will open these files, so Mac users are out of luck. There are some kludges for both of these problems, but they suck. For one thing, none of them preserve the advanced features of the original .docx file – they usually produce RTF documents, so things like “Track Changes” will be stripped out.

What’s the impact? Well, to cite two recent examples, a couple of weeks ago I spent the majority of a day emailing different files back and forth with one editor, just to settle on a format that would work on both of our systems. And yesterday, someone actually resorted to printing and faxing a Word document to me, with changes penciled into the margins. Weren’t we supposed to be beyond this in 2007?

Worse than inconvenience, there’s the security problem. The latest example, reported today in Salon, is this story, about how an 8-year-old extracted classified information from US military documents posted in Word format. All the cutting, pasting, and editing that was supposed to be redacted from the public version of the document was in fact preserved in the file’s “Track Changes” buffer, ready for anyone to uncover with a few mouse clicks. Even if you don’t work with classified military intelligence reports, consider the types of information you handle. I work with embargoed press releases, pre-publication scientific papers, off-the-record comments, and sometimes corporate proprietary data that should never see the light of day. How liable would I be if I emailed an editor an unsecure attachment with that information hidden in plain sight? How liable would they be for insisting on using “Track Changes”? And will anyone ever hold Microsoft accountable for this mess?

Hacking the Tree of Life

Continuing the merger between open source software and bioinformatics, researchers are starting to adopt not only the philosophy, but also the work habits of the open source community. The Phyloinformatics Hackathon, going on this week in North Carolina, mimics the group work sessions of other free software projects:

Leading programmers from as far away as Japan and New Zealand have come to NESCent to build the missing pieces of glue software. Participants include developers of specific phylogenetic software packages, on the one hand, and experts in designing glue software toolkits such as BioPerl and BioJava. “The participants are learning a lot from each other,” says computational biologist Arlin Stoltzfus of the National Institute of Standards and Technology. “The phylogeneticists know what the cutting edge phylogenetic methods are and what users need, while the toolkit developers have lots of experience working with standards that allow different software packages to communicate”.

This particular event focuses on software for phylogenetics, the construction of “tree of life” diagrams that plot relationships between different species or strains of organisms. While categorizing species may sound like a purely academic exercise, it’s actually critical for everything from tracking emerging diseases to measuring the success or failure of conservation efforts.

Scientists are almost religious about the open sharing of data and tools, and even formerly proprietary standards in bioinformatics are now opening up. So how long will it be before amateur scientists and high school science fair participants start doing sophisticated genomic analyses on their home computers?