org-grep — Kind of M-x rgrep adapted for Org mode

This tool allows for grepping files in a set of Org directories, formatting the results as a separate Org buffer. This buffer is assorted with a few specific navigation commands so it works a bit like M-x rgrep.

Installation

This tool has been developed on Linux, and likely requires a Unix-like system. Otherwise, one needs compatible find and grep tools, and a shell able to properly decipher the arguments and establish a pipe.

To install org-grep, just copy org-grep.el somewhere Emacs may find it. Optionally, assign some key binding to trigger the tool. For one, I added these lines to my ~/.emacs file:

(autoload 'org-grep "org-grep" nil t)
(define-key org-mode-map "\C-cog" 'org-grep)

yet of course, one may choose any other key binding.

Usage

A few Emacs variables may be set prior to, or after loading org-grep. These are:

org-grep-directories

This is a list of directories which the org-grep command recursively searches. When nil, which is the default, the command searches within the hierarchy identified by the Org standard org-directory variable.

org-grep-extensions

This is a list of file extensions to retain for the search, including the leading period. The default is a list containing the .org string as its sole member. If set to nil, all files are going to be searched, whatever their extension may be.

org-grep-extra-shell-commands

This is a list of Emacs Lisp functions provided by the user, meant to further customize searching. Such functions may be used whenever variables org-grep-directories and org-grep-extensions above are not sufficient to describe user needs. The default is nil, meaning that there is no extra searching. Each element in the list is the symbol name for the function. Each function receives the regular expression given to org-grep, and returns a string holding a shell command to provide some grep-like output. See at the end of this section for an example.

To use this tool, either call M-x org-grep or the keybinding put aside for it, and reply to the Org grep? prompt with a regular expression to search for. Happily enough, Emacs and the grep command use rather similar syntax for regular expressions.

Be well aware that all currently opened files in Emacs are automatically saved to disk before that command gets executed.

This will create an Org buffer with the found lines, each preceded by the base name of the file containing the line, and the line number within that file. The buffer is read-only, but standard Org commands which do not modify the buffer apply, including of course those able to follow links. A few extra commands are also available:

C-c C-c

For the search hit as identified by the position of the cursor, open the corresponding original file (unless it is already visited, of course), make it the current window, with the cursor left on that line.

C-x `

Move to the next search hit, open the corresponding original file, make it the current window, with the cursor left on the original found line.

.

For the search hit as identified by the position of the cursor, open the corresponding original file with the cursor positioned on the original found line. Leave the cursor within the search results window (but see Caveats below).

n

Move to the next search hit, open the corresponding original file with the cursor positioned on the original found line. Leave the cursor within the search results window (but see Caveats below).

p

Move to the previous search hit, open the corresponding original file with the cursor positioned on the original found line. Leave the cursor within the search results window (but see Caveats below).

g

Save all modified files to disk, then refresh the search hit buffer from the actual contents of the disk files.

c

Copy the *Org grep* buffer into another Emacs buffer whose name depends on the regular expression. This effectively saves the buffer contents before starting a new search. The saved copy is a regular Org file with no specially added command. Also, for convenience, all list items are turned into checklist items.

q

Quit the *Org grep* window, deleting it.

In all Org buffers, command C-x ` uses the contents of an existing *Org grep* buffer for moving to the next search hit. If that buffer does not exist, or if there is no following hit, the standard Emacs action is used instead: usually moving to the next compilation error.

Here is an example of using org-grep-extra-shell-commands. Let’s assume that one wants to also search all non-Org files under Git control in the repository located at ~/share/bin/. One could use:

(setq org-grep-extra-shell-commands '(fp-org-grep-in-share-bin))

(defun fp-org-grep-in-share-bin (regexp)
  (concat "(cd ~/share/bin; git grep -n "
          (shell-quote-argument regexp)
          " | sed 's,^,~/share/bin/,')"))

Purpose, history

Switching to Org, I immediately populated hundreds of Org files with data previously accumulated either as Emacs allout files (or Vim!), Tomboy notes or Workflowy items. The standard Org mechanics for searching a collection of files requires them under the control of the Org agenda. Given my volume of notes, Org mode was crawling, so I had to relax the agenda and quickly develop some other mean for searching.

The first org-grep I wrote was based on Emacs standard M-x rgrep, using hooks and other tricky machinery so it works the way I wanted. Yet, M-x rgrep is limited to a single directory. Moreover, the *grep* buffer does not render Org lines as nicely as Org mode does, and this became critical for some long Org lines using a lot of heavy markup.

So I rewrote org-grep with the resulting output as a genuine Org file. This seems like a cleaner and easier way to proceed.

Caveats

org-grep is constantly useful to me, yet a few minor problems remain, which I can easily live with. Here are those I’m aware of:

• The cursor does not come back into the resulting buffer, for some navigation commands meant so it does. (save-current-buffer ...) or (save-excursion ...), or even more explicit handling, all fail to bring the cursor back into the current window, seemingly whenever an Org link gets followed within the Lisp form.
• Navigation commands should reveal the goal line in the original Org buffer containing the grep hit, but the line stays collapsed and hidden. It seems that (org-reveal) does not do its job.
• The search string may not be always highlighted in the resulting buffer, depending on its capitalization. This is because case-fold-search is ignored by the highlighting mechanism in Emacs. The first letter of the pattern is recognized in both cases, this slightly alleviates the problem, this does not work for letters outside ASCII.
• It would be nice to highlight the search pattern in the original Org buffers containing grep hits.