Unifying annotation and freeform text editing for augmented sensemaking
Matthew Siu and Andy Matuschak; June 2024
This is a preview of an unpublished report. Please do not share it.
You’re in the middle of a large project, and you’re stuck. You’re staring at a pile of long, unstructured documents (journals, lab notebooks, meeting notes) with a sense that the clues you need are buried somewhere within. But there’s simply too much to keep in your head as you scroll. Instead, you hatch a plan. If you could find the important snippets and gather them into one place, it would be easier to see everything at once, to notice patterns, and to figure out what to do no ext.
So you create a new working document, and as you read through your unstructured notes, you copy and paste key passages into it. But this process is cumbersome and disorienting: you’re constantly moving both your attention and your cursor between these documents, deciding where to put each new snippet, losing your place in your unstructured notes. The snippets in your working document are isolated from their original context, so it can be hard to remember what each means and why it matters. And as snippets pile up, your fresh document can easily produce the same kind of overwhelm you were trying to escape.
If you’d been working with books or academic papers, rather than your own notes, you might have used annotation tools instead. It’s easy to enter a satisfying flow with these tools. When some phrase strikes you as important, a highlighter lets you capture that reaction immediately, without shifting your focus. Likewise, marginal comments let you record observations as they occur to you, in context. The trouble with these tools is that your annotations end up trapped in the pages of the original documents. But when you’re trying to make sense of a confusing situation, you need to get everything into one place, where you can see, rearrange, and elaborate the pieces into a new whole.
Annotation tools feel great to use, but they don’t support the follow-up thinking you need to do. Text editors give you a flexible canvas for making sense of snippets, but their design is often cumbersome and disorienting when used in this way. If you could move fluidly between these tools, you could use each where it excels and, perhaps, get the best of both worlds. In this paper, we present Latticework, a system which unifies annotation with freeform text editing, in the context of personal knowledge management tools.
Latticework is built to support the workflow we described in the introduction. You forage through messy source documents, accumulating key snippets into a working document for sensemaking (i.e. rearranging and elaborating that material for insight). But, as prior work has described, this process isn’t linear. It’s often convenient to do a bit of preliminary sensemaking in the midst of foraging; conversely, observations you uncover during sensemaking will often lead to another round of foraging, and so on, in a loop.
Latticework’s main goal, then, is to enable fluid movement between these foraging and sensemaking stances. By extension, that means fluid movement between acting on source documents (which emphasize foraging) and on your working document (which emphasizes sensemaking). Ideally, you should be able to shift your focus as it makes sense in the moment, and the work you do in each place should remain visible in the other.
In the introduction, we described two different workflows people might use to capture what’s important from a messy pile of material: highlighting key snippets in a source document, and copying key snippets into a working document. Highlighting is more convenient when foraging, while copying prepares a manipulable canvas for sensemaking. Latticework unifies these operations, so that you can use whichever is most convenient in the moment, while retaining the benefits of the other.
When you highlight a snippet in a source document, a linked copy of that snippet is inserted into your working document, open in the opposite pane. Then, when you shift to sensemaking, you can rearrange the link within that document as you would any ordinary text.
Once you’ve done some sensemaking, it may be more convenient to use the other workflow. Suppose that you see an important snippet as you scan through a source document, and you know just where it should go in your emerging structure. If you highlight the snippet, a link will be inserted at the bottom of the working document, which isn’t quite what you want. Instead, you can select the snippet, invoke the “Copy Snippet Link” command (Command+Option+C), and paste into the working document wherever you’d like. Then—unlike in a typical copy-and-paste workflow—you’ll see that snippet highlighted in the source document, so that you can keep track of your work as you continue reading.
You’ll get the same result no matter which direction you go—a highlight in the source document and a snippet link in the working document. Conceptually, highlighting doesn’t actually modify the source document. Highlights are a dynamic style applied to all the snippets linked in your working document. So if you delete a snippet link, the corresponding highlight will disappear, too.
While you’re gathering these snippets, you may also want to capture observations about them. Each workflow has a natural way to handle this. If you’re reading a source document with a highlighter, you can write comments in the margins. If you’re copying snippets into a working document, you can type observations alongside them. As with highlighting and copying, Latticework makes these operations interchangeable.