01.04.2023 at 11:27 am

Search Notes Fast: With Taskwarrior, jq & ripgrep

Search through Taskwarrior annotations fast.

The Code

If you've installed Taskwarrior, jq and ripgrep, you're already good to go! The searching/filtering command is actually a single line:

task export | jq -r '.[].annotations[]?.description' | rg  -ni

I prefer to alias the full command (I like tt because of how short it is):

alias tt="(above)"

See the breakdown below!

Why/How I Take Notes & Annotate

As a lawyer, I write notes all the time.

And I do it with perhaps more vigour than most people, given the weight of documentation and the professional care taken in my line of work.

I've therefore a bit of a ritual:

Making/Searching Annotations with Taskwarrior

Given how important my notes are to me, two points arise: How fast can I make them? And how do I retrieve them at speed?

I previously used Todoist. It was painful for annotations, and terrible at searching notes. I've stopped using it personally, and no longer recommend it.

These days? I use Taskwarrior. I love just how fast it lets me make and annotate my tasks. (The default key bindings - a to add tasks, and A to annotate - work great.) It blazes when searching, but generates clutter when sometimes all I want are just my notes.

That led me to think: what if I pair Taskwarrior with jq and ripgrep? Will it work?

I'm pleased to report: yes! Frankly, this is the fastest system I've ever used, which counts for much if you're a prolific note-taker/note-searcher like me.

Here's an example, if I were to look for annotations on 'vim', 'rust' or 'zombie' in my tasklist:

Requirements: jq / ripgrep

If you've never used jq and ripgrep:

  1. jq is a data processor of sorts. It works with JSON, and lets you handle compatible data structures from the command line in an almost object-oriented-like fashion.

  2. ripgrep is a text-searching command line tool. It lets you search through files/directories for specific text patterns. You can also feed it lines of text.

The Thought Process

The command, when broken-down, has three parts. Let's go step by step:

  1. First, export all tasks with Taskwarrior. On the commandline:

    > task export

    The command task export gets you JSON data/dicts, line by line.

  2. Process with jq. Each line contains a key-value structure titled annotations, containing two keys - entry (date-time) and description (the annotation-text). You only want the latter, so pipe the exports through jq:

    > ... | jq -r '.[].annotations[]?.description'

    Notice the ? symbol above. Sometimes jq can't read Taskwarrior's exported lines (which gets you Cannot iterate over null errors). The ? is a validity/exception check; it lets processing continue past erroneous lines.

    And here's a more detailed variant with UUIDs, via string interpolation:

    > ... | jq -r '.[] |  \"\(.uuid) / \(.annotations[]?.description)\"'
  3. Search with ripgrep. Once jq returns with all your annotations, hit it with ripgrep to filter for lines matching the phrase. Add as much salt as you wish; I prefer just a pinch of the basics - case insensitivity, line numbers:

    > ... | rg  -ni

The above is the perfect minimalist solution for me, as I'm only interested in the content of my notes. But with a bit of smarts, you could expand upon this into other cool ideas - e.g. annotations with entry dates, UUIDs, and so on.

Performance Concerns

One more consideration: I do not pre-process or filter tasks. I prefer retrieving all tasks outright (via task export).

Am I worried about performance? Not at all! First, I see no need to optimize prematurely:

... programmers have spent far too much time worrying about efficiency in the wrong places and at the wrong times; premature optimization is the root of all evil (or at least most of it) in programming.

- Donald Knuth

Taskwarrior is also performant - it can go through thousands of tasks on the order of microseconds. Given how insanely fast modern/future CPUs are:

  1. If you have no scalability issues, there's little to worry about.

  2. Even with the overhead of jq + ripgrep, Taskwarrior slowdowns won't be felt for single users.

  3. Even if I had 10x the amount of notes I now have (which will take years or decades to amass), CPU improvements would have scaled alongside it in the meantime.

Filed under:
Words: 993 words approx.
Time to read: 3.97 mins (at 250 wpm)
, , , , , , , , ,

Other suggested posts

  1. 07.04.2021 at 12:00 am / Harangued Commitments
  2. 29.09.2020 at 12:24 pm / Rays of Circumstance
  3. 25.08.2020 at 08:42 am / Triple Buffered Painters
  4. 02.01.2020 at 11:48 am / Theory and Theatre
  5. 31.12.2018 at 06:23 pm / Why Plain Text
  6. 15.12.2017 at 12:00 am / The Python Paradox
  7. 28.06.2015 at 12:00 am / No Social Transformation Without Representation
  8. 17.03.2012 at 12:00 am / Upper and Lower Hengsha
  9. 24.08.2010 at 12:00 am / Correspondences With Matt Treyvaud
  10. 22.08.2010 at 12:00 am / 柿が赤くなると医者が青くなる
© Wan Zafran. See disclaimer.