emacs.d/README.org
2022-01-13 16:22:33 +01:00

105 KiB
Raw Blame History

Emacs setup for use with LaTeX, Org, and Python

Quick start

Backup your user-emacs-directory (defaults often to ~/.emacs.d on Linux, Unix, or Darwin) directory and execute the commands in listing lst:prepare-user-emacs-directory. After invoking Emacs interactively (in interactive mode, neither in batch mode, nor in server mode), Emacs will ask you to install a selected set of packages. Quit Emacs and invoke Emacs again.

  cd ~
  git clone ccdr@mercury.grenoble.cnrs.fr:SERVER/emacs.d.git .emacs.d
  make --directory=.emacs.d init
  emacs &

Introduction

This Emacs setup aims to install automatically a minimal set of extension packages that allows to handle my reports and presentations. The file format of the reports is Org Mode plain text with Python source code blocks and the file format of the presentations is LaTeX.

This org file (more precisely the original org source file of this file) illustrates three methods in my work-flow:

  1. How to tangle (or export) source blocks from org files. This file contains source blocks to produce the files early-init.el, init.el, latexmkrc, org-store-link, and example.py by tangling.
  2. How to export org files to other formats such as HTML, LaTeX, and PDF.
  3. How org hyperlinks (info) allow to link inside and outside Org Mode: hover over or click on the links to experiment.

The AUCTeX - Aalborg University Center TeX extension package provides a powerful Text-based User Interface (TUI) environment to edit the LaTeX presentations.

The citar extension package provides quick filtering and selecting of bibliographic entries, and the option to run different commands on those selections. Citar requires Org-9.5 (info), which is already part of Emacs-28.1. Citar exploits the enhancements of Emacs' builtin selection mechanism provided by the extension packages vertico, orderless, embark, marginalia, and consult. The citeproc extension package provides CSL: citation style language processing capabilities to citar and Org Mode.

The pdf-tools extension package renders PDF file with the possibility to annotate the file or to click on anchors in the PDF file that link back to the original LaTeX file of the document. An example of my work-flow are the steps to convert this org file to PDF and to see the result with pdf-tools in Emacs: execute the commands pdf-tools-install, org-babel-tangle, org-latex-export-latex-to-latex, and compile. This sets up an infinite LaTeX compilation loop to update and redisplay the PDF file after excution of the org-latex-export-latex-to-latex command in this buffer.

Here follows a list of interesting Emacs configurations:

  1. Musa Al-hassy's configuration is an impressive example of producing the Emacs initialization files and other files by tangling an org file. His methodology is impressive, as his Elisp Cheat Sheet and org-special-block-extra package show. To me, this is a configuration to admire, but his methodology is way over my head.
  2. Omar Antolín Camarena's configuration exploits built-in packages, Omar's own small packages, and large external packages. Omar is the author of orderless and embark. I have stolen his idea of using custom-set-variables.
  3. Pierre Neirhardt's configuration implements lazy loading without help of external packages. I have stolen his approach of using lazy loading to silently ignore the setup stanzas of uninstalled extension packages.
  4. Sacha Chua's configuration is a practical example of producing the Emacs initialization files by tangling an org file. It gives me the impression that she is a very practical person trying to achieve her goals by the most efficient means. I have stolen her idea of using quelpa to install packages from any source.
  5. Steve Purcell's configuration is well organized, a showcase of readable code, as well helpful commit and issue histories. See for instance the discussion on the correctness of order of company candidates in Emacs lisp mode.

Early Init File (info)

Try to load no-littering as early as possible, since it helps to keep ~/.emacs.d clean.

  ;;; early-init.el --- user early-init file        -*- lexical-binding: t -*-
  ;;; Commentary:
  ;;; Code:
  (setq load-prefer-newer t)

  (require 'no-littering nil 'noerror)

  (provide 'early-init)
  ;; Emacs looks for "Local variables:" after the last "?\n?\f".
  
  ;; Local Variables:
  ;; indent-tabs-mode: nil
  ;; End:
  ;;; earl-init.el ends here

In order to get help in understanding the code block above in a buffer showing the original Org source file, type after moving point (or cursor) to one of the items of the list:

  1. src_emacs-lisp{(describe-variable #'load-prefer-newer t)}
  2. src_emacs-lisp{(apropos-library "no-littering")}
  3. src_emacs-lisp{(find-function #'hack-local-variables)}

to execute the code between the curly braces for access to help. This shows that Emacs is a self-documenting editor.

Init File (info) header

The user-init-file header requires cl-lib and customizes Emacs variables. It consists of three parts in listing lst:1st-custom-set-variables-call, lst:2nd-custom-set-variables-call, and lst:3rd-custom-set-variables-call in order to limit the length of the listings for exporting to LaTeX. The quoting (info) and the backquote (info) pages explain how to understand the reader macros ' (quote), ` (backquote), , (substitute) and @, (splice) in the custom-set-variables function calls in listing lst:1st-custom-set-variables-call, lst:2nd-custom-set-variables-call, and lst:3rd-custom-set-variables-call. A tutorial of how to use those reader macros is the didactic emacs-lisp macro example. The init file (info) does not load the custom-file as saving customizations (info) recommends because of the custom-set-variables function calls.

  ;;; init.el --- user init file                    -*- lexical-binding: t -*-
  ;;; Commentary:
  ;;; Code:
  (require 'cl-lib)

  (custom-set-variables
   '(after-save-hook #'executable-make-buffer-file-executable-if-script-p)
   '(column-number-mode t)
   '(cursor-type 'box)
   `(custom-file ,(locate-user-emacs-file "custom.el"))
   '(epg-pinentry-mode 'loopback)
   '(global-hl-line-mode t)
   '(global-hl-line-sticky-flag t)
   '(indent-tabs-mode nil)
   '(inhibit-startup-buffer-menu t)
   '(inhibit-startup-screen t)
   '(initial-buffer-choice t)
   '(initial-scratch-message "")
   `(insert-directory-program ,(or (executable-find "gls")
                                   (executable-find "ls")))
   '(kill-ring-max 300)
   '(package-archives '(("gnu" . "https://elpa.gnu.org/packages/")
                        ("nongnu" . "https://elpa.nongnu.org/nongnu/")
                        ("melpa" . "https://melpa.org/packages/")))
   ;; Pin those packages to GNU ELPA to get the info documentation.
   '(package-pinned-packages '((consult . "gnu")
                               (marginalia . "gnu")
                               (vertico . "gnu"))))
  (custom-set-variables
   `(package-selected-packages
     `(,@(when (version< emacs-version "28.0")
           '(org))                ; plain text thought organizer
       anaconda-mode              ; strangles python-mode
       async                      ; asynchroneous processing
       auctex                     ; Aalborg University Center TeX
       blacken                    ; Black Python-code formatter client
       citar                      ; bibliography handling
       citeproc                   ; bibliography handling
       company                    ; complete anything
       company-anaconda           ; complete anything in anaconda-mode
       consult                    ; consult completing-read
       eglot                      ; Emacs polyGLOT LSP client
       electric-operator          ; automatic spacing around operators
       elfeed                     ; web feed reader
       embark                     ; act on any buffer selection
       emms                       ; Emacs Multi-Media System
       htmlize                    ; convert buffer contents to HTML
       iedit                      ; simultaneous multi-entity editing
       laas                       ; LaTeX Auto-Activating Snippets
       magit                      ; Git Text-based User Interface
       marginalia                 ; minibuffer margin notes
       markdown-mode              ; markdown text mode
       no-littering               ; keep `user-emacs-directory' clean
       nov                        ; EPUB reader
       orderless                  ; Emacs completion style
       pdf-tools                  ; interactive docview replacement
       pdf-view-restore           ; add view history to pdf-tools
       pyenv-mode                 ; Python environment selector
       quelpa                     ; install Emacs packages from source
       rainbow-mode               ; set background color to color string
       smartparens                ; smart editing of character pairs
       toml-mode                  ; Tom's Obvious Minimal Language mode
       undo-tree                  ; more advanced yet simpler undo system
       vertico                    ; VERTical Interactive Completion
       wgrep                      ; open a writable grep buffer
       which-key                  ; on the fly key-binding help
       wordnut                    ; WordNet lexical database
       writegood-mode             ; bullshit and weasel-word detector
       ws-butler                  ; remove trailing whitespace
       xr                         ; undo rx to grok regular expressions
       yasnippet)))               ; code or text template expansion
  (custom-set-variables
   '(recentf-mode t)
   '(save-place-mode t)
   '(scroll-bar-mode nil)
   '(tab-always-indent 'complete)
   '(tab-width 8)
   '(tool-bar-mode nil)
   '(url-cookie-trusted-urls nil)
   '(url-cookie-untrusted-urls '(".*"))
   '(use-dialog-box nil)
   '(use-short-answer t)
   '(view-read-only t))

  (when (eq system-type 'darwin)
    (custom-set-variables
     '(ns-alternate-modifier nil)
     '(ns-command-modifier 'meta)
     '(ns-right-command-modifier 'super)))

  (when (eq window-system 'ns)
    (add-to-list 'initial-frame-alist '(height . 51))
    (add-to-list 'initial-frame-alist '(width . 180)))

Install the selected packages (info)

Emacs installs packages from archives on the internet. This setup uses three archives:

  1. The GNU Emacs Lisp Package Archive
  2. The NonGNU Emacs Lisp Package Archive.
  3. The Milkypostmans Emacs Lisp Package Archive (MELPA).

In addition, the quelpa tool allows to fetch code from any source and build a package on your computer before installation. The code in listing lst:install-selected-packages assumes that the package system is in a virgin state if the package no-littering is not present:

  1. It installs and loads no-littering after ensuring refreshing of the contents of available packages.
  2. It ensures installation of quelpa before ensuring installation of undo-tree.
  3. It calls src_emacs-lisp{(package-install-selected-packages)} to check the installation status of all packages in src_emacs-lisp{package-selected-packages} and to install the missing packages after the user has agreed to its prompt.

In case of normal Emacs usage, src_emacs-lisp{(package-list-packages)} refreshes the contents of packages and allows to update packages to the latest version.

  (unless (require 'no-littering nil 'noerror)
    (package-refresh-contents)
    (package-install 'no-littering)
    (require 'no-littering))

  (unless (package-installed-p 'quelpa)
    (package-install 'quelpa))

  (unless (package-installed-p 'undo-tree)
    ;; Neither GNU ELPA, nor MELPA have the latest version.
    (quelpa '(undo-tree :repo "tsc25/undo-tree" :fetcher gitlab)))

  (unless noninteractive
    (package-install-selected-packages))

Info documentation

Listing lst:configure-info fixes what looks like a bug in Emacs-28.0.91 and adds a path in my home directory to the places where info looks for files.

  (with-eval-after-load 'info
    (unless (version< emacs-version "28.0")
      ;; Why is this necessary after `package-activate-1'?
      (dolist (item package-alist)
        (let ((pkg-dir (package-desc-dir (cadr item))))
          (when (file-exists-p (expand-file-name "dir" pkg-dir))
            (cl-pushnew pkg-dir Info-directory-list :test #'equal)))))

    ;; Emacs should find my "python.info" file.
    (add-to-list 'Info-directory-list
                 (expand-file-name "~/.local/share/info")))

Using Emacs as a server (info)

Emacs can act as a server that listens to a socket to share its state (for instance buffers and command history) with other programs by means of a shell command emacsclient. Section #sec:latexmk-save-compile-display-loop and #sec:qutebrowser-userscript show how to use emacsclient to:

  1. Install an asynchronous (or background) loop of saving a LaTeX file, compiling it, and redisplaying the output in Emacs.
  2. Make qutebrowser send html links with document titles to Emacs.

The code in listing lst:start-emacs-server starts the Emacs server.

  (when window-system
    (unless (or noninteractive (daemonp))
      (add-hook 'after-init-hook #'server-start)))

Latexmk save-compile-display loop

The latexmk resource file in the next source code block shows how to use emacsclient to (re)display the PDF file in Emacs after each succesful (re)compilation on condition that the settings of the compile-command local variable in section are compatible. The local variable compile-command in the local variables section (only visible in org files, but not in html and pdf files) shows how to use the latexmkrc file.

  # pdf previewer and update pdf previewer
  $pdf_previewer = "emacsclient -e '(find-file-other-window %S)'";
  $pdf_update_method = 4;  # 4 runs a command to force the update
  $pdf_update_command = "emacsclient -e '(with-current-buffer (find-buffer-visiting %S) (pdf-view-revert-buffer nil t))'";
  # see for instance glossary.latexmkrc
  add_cus_dep( 'acn', 'acr', 0, 'makeglossaries' );
  add_cus_dep( 'glo', 'gls', 0, 'makeglossaries' );
  $clean_ext .= " acr acn alg bbl glo gls glg ist lol run.xml";
  sub makeglossaries {
      my ($name, $path) = fileparse( $$Psource );
      return system "makeglossaries -d '$path' '$name'";
  }
  # Emacs looks for "Local variables:" after the last "?\n?\f".
  
  # Local Variables:
  # mode: perl
  # End:

Qutebrowser userscript

The next block contains an userscript that sends a store-link org-protocol message with the url and the title from qutebrowser to emacsclient. The function urlencode translates the url and the title for the message. The Python urllib examples show how to use urlencode. The final execvp call deals with a qutebrowser userscript requirement: the emacsclient process must get the PID of the userscript that must kill itself after the take-over. Termination of the emacsclient process hands control back to qutebrowser.

On a POSIX system, you can run the userscript from qutebrowser or from a terminal to see whether it works. In case you try to run it from Emacs, Emacs may hang or die.

  #!/usr/bin/env python
  from urllib.parse import urlencode
  from os import environ, execvp

  url = environ.get("QUTE_URL", "https://orgmode.org")
  title = environ.get("QUTE_TITLE", "Org Mode")
  parameters = urlencode({"url": url, "title": title})
  print(payload := f"org-protocol://store-link?{parameters}")
  execvp("emacsclient", ("-n", payload))

TODO Look into: org-protocol handling with other browser on Darwin

Completion

Vertico (info) provides a performant and minimalistic vertical completion UI based on the default completion system and behaves therefore correctly under all circumstances. Using Vertico, Marginalia, Consult, and Embark links to a video demonstration. Vertico integrates well with fully supported complementary packages to enrich the completion UI:

  1. Orderless (info) for an advanced completion style,
  2. Embark (info) for minibuffer actions with context menus,
  3. Marginalia (info) for rich annotations in the minibuffer, and
  4. Consult (info) for useful search and navigation commands,

where the order is that of enhancing citar's experience and the configuration steps below.

Finally, company: a modular complete-anything framework for Emacs provides completion in any buffer and minibuffer-history-completion provides completion on previous input in the minibuffer.

Vertico (info)

Listing lst:enable-vertico-mode configures and enables savehist-mode and enables vertico-mode. The documentation src_emacs-lisp{(describe-function 'savehist-mode)} why it is best to turn on savehist-mode in the Emacs init file.

  (unless noninteractive
    (custom-set-variables
     '(history-delete-duplicates t)
     '(history-length 500)
     '(savehist-additional-variables
       '(eww-history
         kill-ring
         regexp-search-string
         search-ring
         search-string)))
    (savehist-mode +1)
    (when (fboundp 'vertico-mode)
      (vertico-mode +1))
    (with-eval-after-load 'vertico
      (define-key vertico-map (kbd "RET") #'vertico-directory-enter)
      (define-key vertico-map (kbd "DEL") #'vertico-directory-delete-char)
      (define-key vertico-map (kbd "M-DEL") #'vertico-directory-delete-word)))
command keys remap
vertico-directory-delete-char
vertico-directory-delete-word
vertico-directory-enter
vertico-exit exit-minibuffer
vertico-exit-input
vertico-first beginning-of-buffer
vertico-first minibuffer-beginning-of-buffer
vertico-insert
vertico-last end-of-buffer
vertico-next-group forward-paragraph
vertico-next next-line-or-history-element
vertico-next next-line
vertico-previous-group backward-paragraph
vertico-previous previous-line-or-history-element
vertico-previous previous-line
vertico-save kill-ring-save
vertico-scroll-down scroll-down-command
vertico-scroll-up scroll-up-command

Orderless (info)

Listing lst:configure-orderless enables orderless.

  (unless noninteractive
    (when (fboundp 'orderless-filter)
      (custom-set-variables
       ;; https://github.com/purcell/emacs.d/issues/778
       '(completion-styles '(basic completion-partial orderless))
       '(completion-category-defaults nil)
       '(completion-category-overrides
         '((file (styles partial-completion)))))
      (add-hook 'minibuffer-setup-hook
                (defun my-on-minibuffer-setup-hook()
                  (setq-default completion-styles '(substring orderless))))))

Embark (info)

Listing lst:configure-embark configures embark.

  (unless noninteractive
    (when (cl-every #'fboundp '(embark-act embark-bindings embark-dwim))
      (global-set-key (kbd "C-,") #'embark-act)
      (global-set-key (kbd "C-:") #'embark-dwim)
      (global-set-key (kbd "C-h B") #'embark-bindings)))

Marginalia (info)

Listing lst:enable-marginalia-mode enables marginalia-mode.

  (unless noninteractive
    (when (fboundp 'marginalia-mode)
      (marginalia-mode +1)))

Consult (info)

Listing lst:configure-consult configures consult.

command keys key map
consult-apropos global-map
consult-bookmark ctl-x-keymap
consult-buffer-other-frame ctl-x-keymap
consult-buffer-other-window ctl-x-keymap
consult-buffer ctl-x-keymap
consult-compile-error goto-map
consult-complex-command ctl-x-keymap
consult-find search-map
consult-focus-lines search-map
consult-git-grep search-map
consult-global-mark goto-map
consult-goto-line goto-map
consult-goto-line goto-map
consult-history global-map
consult-imenu-project goto-map
consult-keep-lines search-map
consult-line search-map
consult-mark goto-map
consult-mode-command global-map
consult-multi-occur search-map
consult-outline goto-map
consult-register ctl-x-keymap
consult-yank-pop global-map
elfeed global-map
embark-act global-map
embark-bindings global-map
embark-dwim global-map
iedit-mode global-map
minibuffer-complete-history minibuffer-local-map
narrow-or-widen-dwim ctl-x-keymap
org-agenda global-map
org-capture global-map
org-cite org-mode-map
org-insert-link-global global-map
org-narrow-to-table ctl-x-keymap
org-store-link global-map
  (unless noninteractive
    (when (fboundp 'consult-apropos)
      (custom-set-variables
       '(consult-project-root-function #'vc-root-dir))
      ;; C-c bindings (mode-specific-map)
      (global-set-key (kbd "C-c h") #'consult-history)
      (global-set-key (kbd "C-c m") #'consult-mode-command)
      ;; C-x bindings (ctl-x-map)
      (define-key ctl-x-map (kbd "M-:") #'consult-complex-command)
      (define-key ctl-x-map (kbd "b") #'consult-buffer)
      (define-key ctl-x-map (kbd "4 b") #'consult-buffer-other-window)
      (define-key ctl-x-map (kbd "5 b") #'consult-buffer-other-frame)
      (define-key ctl-x-map (kbd "r x") #'consult-register)
      (define-key ctl-x-map (kbd "r b") #'consult-bookmark)
      ;; M-g bindings (goto-map)
      (define-key goto-map (kbd "g") #'consult-goto-line)
      (define-key goto-map (kbd "M-g") #'consult-goto-line)
      (define-key goto-map (kbd "o") #'consult-outline)
      (define-key goto-map (kbd "m") #'consult-mark)
      (define-key goto-map (kbd "k") #'consult-global-mark)
      (define-key goto-map (kbd "i") #'consult-imenu-project)
      (define-key goto-map (kbd "e") #'consult-compile-error)
      ;; M-s bindings (search-map)
      (define-key search-map (kbd "g") #'consult-git-grep)
      (define-key search-map (kbd "f") #'consult-find)
      (define-key search-map (kbd "k") #'consult-keep-lines)
      (define-key search-map (kbd "l") #'consult-line)
      (define-key search-map (kbd "m") #'consult-multi-occur)
      (define-key search-map (kbd "u") #'consult-focus-lines)
      ;; Other bindings
      (global-set-key (kbd "M-y") #'consult-yank-pop)
      (global-set-key (kbd "<help> a") #'consult-apropos)
      ;; Tweak functions
      (advice-add 'completing-read-multiple
                  :override #'consult-completing-read-multiple)
      (fset 'multi-occur #'consult-multi-occur)))

Company: a modular complete anything framework for Emacs

Listing lst:configure-company configures company.

  (unless noninteractive
    (when (fboundp 'company-mode)
      (custom-set-variables
       ;; https://github.com/purcell/emacs.d/issues/778
       '(company-transformers '(company-sort-by-occurrence)))
      (dolist (hook '(LaTeX-mode-hook
                      org-mode-hook
                      emacs-lisp-mode-hook
                      lisp-interaction-mode-hook
                      python-mode-hook
                      ielm-mode-hook))
        (add-hook hook #'company-mode))))

Minibuffer history completion

See Juri Linkov (Emacs Developer mailing list) for how to allow completion on previous input in the minibuffer. Listing lst:enable-minibuffer-history-completion enables minibuffer history completion.

  (defun minibuffer-setup-history-completions ()
    (unless (or minibuffer-completion-table minibuffer-completion-predicate)
      (setq-local minibuffer-completion-table
                  (symbol-value minibuffer-history-variable))))

  (add-hook 'minibuffer-setup-hook 'minibuffer-setup-history-completions)

  ;; Stolen from Emacs-28.1 for Emacs-27.2:
  (unless (fboundp 'minibuffer--completion-prompt-end)
    (defun minibuffer--completion-prompt-end ()
      (let ((end (minibuffer-prompt-end)))
        (if (< (point) end)
            (user-error "Can't complete in prompt")
          end))))

  ;; Adapted from minibuffer-complete:
  (defun minibuffer-complete-history ()
    "Allow minibuffer completion on previous input."
    (interactive)
    (completion-in-region (minibuffer--completion-prompt-end) (point-max)
                          (symbol-value minibuffer-history-variable)
                          nil))

  (define-key minibuffer-local-map (kbd "C-<tab>") #'minibuffer-complete-history)

Prefix key-binding help

Configure which-key-mode so that typing C-h after a prefix key displays all keys available after the prefix key. Listing lst:enable-which-key-mode enables manual activation of which-key-mode. This uses prefix-help-command behind the scenes and is therefore incompatible with straightforward use of [[info:embark#Quick start][=embark-prefix-help-command= (info)]].

  (when (fboundp 'which-key-mode)
    (custom-set-variables
     '(which-key-idle-delay 10000)
     '(which-key-idle-secondary-delay 0.05)
     '(which-key-show-early-on-C-h t))
    (which-key-mode +1))

Reading

Reading EPUB files

Listing lst:enable-nov-mode enables nov-mode.

  (when (fboundp 'nov-mode)
    (add-to-list 'auto-mode-alist `(,(rx ".epub" eos) . nov-mode)))

Reading PDF files

The pdf-tools package exploits the poppler library to render and to let you annotate PDF files. It also exploits the SyncTeX library to link anchors in PDF files produced with LaTeX to the original LaTeX sources.

In order to use pdf-tools, you have to type M-x pdf-tools-install after installation of pdf-tools from MELPA or after each update of poppler to build or rebuild the epdfinfo executable that serves the PDF files to Emacs.

  ;; 'pdf-loader-install' is the lazy equivalent of 'pdf-tools-install':
  ;; see the README file.
  (when (fboundp 'pdf-loader-install)
    (pdf-loader-install))

  (with-eval-after-load 'pdf-view
    (when (fboundp 'pdf-view-restore-mode)
      (add-hook 'pdf-view-mode-hook #'pdf-view-restore-mode)))

Writing

Writing LaTeX files

Loading tex.el immediately instead of lazily ensures proper initialization of AUCTeX. For instance, the TeX-master safe local variable in the tex.el elisp library file has no autoload cookie. Without prior loading of tex.el, Emacs will complain that TeX-master is no safe local variable in case it reads a LaTeX file that sets TeX-master. Listing lst:require-auctex initializes AUCTeX properly.

  ;; Use `require' to make `TeX-master' a safe local variable.
  (when (require 'tex nil 'noerror)
    (custom-set-variables
     '(TeX-auto-save t)
     '(TeX-install-font-lock #'font-latex-setup)
     '(TeX-parse-self t)))

Although, the LaTeX biblatex is in use, listing lst:configure-bibtex configures the Emacs bibtex library for the LaTeX BiBTeX package to maintain backwards compatibility.

  (with-eval-after-load 'bibtex
    (custom-set-variables '(bibtex-dialect 'BibTeX)))

Listing lst:configure-font-latex disables font scaling of section titles.

  (with-eval-after-load 'font-latex
    (custom-set-variables
     '(font-latex-fontify-sectioning 1.0)))

Listing lst:configure-latex configures latex for a full featured LaTeX-section-command.

  (with-eval-after-load 'latex
    (custom-set-variables
     '(LaTeX-section-hook '(LaTeX-section-heading
                            LaTeX-section-title
                            LaTeX-section-toc
                            LaTeX-section-section
                            LaTeX-section-label))))

Out of the box, AUCTeX does not indent text between square brackets. The code in listing lst:configure-tex corrects this by advising to override TeX-brace-count-line with my-TeX-brace-count-line.

  (with-eval-after-load 'tex
    ;; https://emacs.stackexchange.com/questions/17396/
    ;; indentation-in-square-brackets
    (defun my-TeX-brace-count-line ()
      "Count number of open/closed braces."
      (save-excursion
        (let ((count 0) (limit (line-end-position)) char)
          (while (progn
                   (skip-chars-forward "^{}[]\\\\" limit)
                   (when (and (< (point) limit) (not (TeX-in-comment)))
                     (setq char (char-after))
                     (forward-char)
                     (cond ((eq char ?\{)
                            (setq count (+ count TeX-brace-indent-level)))
                           ((eq char ?\})
                            (setq count (- count TeX-brace-indent-level)))
                           ((eq char ?\[)
                            (setq count (+ count TeX-brace-indent-level)))
                           ((eq char ?\])
                            (setq count (- count TeX-brace-indent-level)))
                           ((eq char ?\\)
                            (when (< (point) limit)
                              (forward-char) t))))))
          count)))
    (advice-add 'TeX-brace-count-line :override #'my-TeX-brace-count-line))

TODO Improve the AUCTeX configuration slowly

Writing Org files

Org activation (info)

  ;; Inspect:
  ;; function with "C-h f"
  ;; symbols with "C-h o"
  ;; variables with "C-h v"

  (global-set-key (kbd "C-c a") #'org-agenda)
  (global-set-key (kbd "C-c c") #'org-capture)
  (global-set-key (kbd "C-c l") #'org-store-link)
  (global-set-key (kbd "C-c C-l") #'org-insert-link-global)

Org customization

The code in listing lst:customize-org-babel, lst:customize-org, and lst:customize-org-export does basic customization of Org mode variables.

  (with-eval-after-load 'ob-core
    (custom-set-variables
     '(org-confirm-babel-evaluate nil)))

  (with-eval-after-load 'ob-python
    (custom-set-variables
     '(org-babel-python-command "python -E")))

  (with-eval-after-load 'ob-latex
    (custom-set-variables
     '(org-babel-latex-preamble
       (lambda (_)
         "\\documentclass[preview]{standalone}\n"))
     '(org-babel-latex-begin-env
       (lambda (_)
         "\\begin{document}\n"))
     '(org-babel-latex-end-env
       (lambda (_)
         "\\end{document}\n"))))
  (with-eval-after-load 'org
    (custom-set-variables
     '(org-babel-load-languages '((C . t)
                                  (calc . t)
                                  (dot . t)
                                  (emacs-lisp . t)
                                  (eshell . t)
                                  (fortran . t)
                                  (gnuplot . t)
                                  (latex . t)
                                  (lisp . t)
                                  (maxima . t)
                                  (org . t)
                                  (perl . t)
                                  (python . t)
                                  (scheme . t)
                                  (shell . t)))
     '(org-export-backends '(ascii beamer html icalendar latex odt texinfo))
     '(org-file-apps '((auto-mode . emacs)
                       (directory . emacs)
                       ("\\.mm\\'" . default)
                       ("\\.x?html?\\'" . default)
                       ("\\.pdf\\'" . emacs)))
     '(org-modules '(ol-bibtex
                     ol-doi
                     ol-eww
                     ol-info
                     org-id
                     org-protocol
                     org-tempo))
     '(org-structure-template-alist '(("a" . "export ascii")
                                      ("c" . "center")
                                      ("C" . "comment")
                                      ("e" . "example")
                                      ("E" . "export")
                                      ("h" . "export html")
                                      ("l" . "export latex")
                                      ("q" . "quote")
                                      ("s" . "src")
                                      ("p" . "src python :session :async")
                                      ("v" . "verse"))))
    (define-key org-mode-map (kbd "M-q") #'org-fill-paragraph))
  (with-eval-after-load 'ox-latex
    (custom-set-variables
     '(org-latex-compiler "lualatex")
     '(org-latex-hyperref-template "\\hypersetup{
    pdfauthor={%a},
    pdftitle={%t},
    pdfkeywords={%k},
    pdfsubject={%d},
    pdfcreator={%c},
    pdflang={%L},
    citecolor=blue,
    colorlinks=true,
    filecolor=blue,
    hyperfootnotes=false,
    linkcolor=blue,
    unicode=true,
    urlcolor=blue,
  }\n")
     '(org-latex-listings 'minted)
     '(org-latex-minted-langs '((cc "c++")
                                (cperl "perl")
                                (diff "diff")
                                (shell-script "bash")
                                (caml "ocaml")
                                (org "text")))
     '(org-latex-minted-options '(("bgcolor" "LightGoldenrodYellow")))
     `(org-latex-logfiles-extensions
       ',(cl-union '("lof" "lot") org-latex-logfiles-extensions :test #'equal))
     '(org-latex-prefer-user-labels t)
     '(org-latex-subtitle-separate t)))

Citar: citing bibliography with Org Mode

Citar is a completing-read front-end to browse and act on BibTeX, BibLaTeX, as well as CSL JSON bibliographic data with LaTeX, markdown, and org-cite editing support. In combination with vertico, orderless, embark, marginalia, and consult, Citar provides quick filtering and selecting of bibliographic entries from the minibuffer, as well as the option to run different commands on those selections. Listing lst:configure-oc-cite+citar configures org-cite, citar, and org.

  (with-eval-after-load 'oc
    (require 'oc-biblatex)
    (require 'oc-csl)

    (custom-set-variables
     '(org-cite-export-processors '((latex biblatex)
                                    (t csl)))
     '(org-cite-global-bibliography '("~/VCS/research/refs.bib")))

    (when (require 'citar nil 'noerror)
      (custom-set-variables
       '(org-cite-activate-processor 'citar)
       '(org-cite-follow-processor 'citar)
       '(org-cite-insert-processor 'citar))))

  (with-eval-after-load 'org
    (when (require 'citar nil 'noerror)
      (custom-set-variables
       '(citar-bibliography '("~/VCS/research/refs.bib"))
       '(citar-file-extensions '("djvu" "pdf"))
       '(citar-library-paths '("~/VCS/research/papers/"))))

    (define-key org-mode-map (kbd "C-c b") #'org-cite-insert))

Translate capital keywords (old) to lower case (new)

  (with-eval-after-load 'org
    (defun org-syntax-convert-keyword-case-to-lower ()
      "Convert all #+KEYWORDS to #+keywords."
      (interactive)
      (save-excursion
        (goto-char (point-min))
        (let ((count 0)
              (case-fold-search nil))
          (while (re-search-forward "^[ \t]*#\\+[A-Z_]+" nil t)
            (unless (s-matches-p "RESULTS" (match-string 0))
              (replace-match (downcase (match-string 0)) t)
              (setq count (1+ count))))
          (message "Replaced %d keywords" count)))))

Evaluate specific source blocks at load-time

How to do load time source block evaluation

  (defun my-org-eval-blocks-named (infix)
    "Evaluate all source blocks having INFIX in their name."
    (when (eq major-mode 'org-mode)
      (let ((blocks
             (org-element-map
                 (org-element-parse-buffer 'greater-element nil) 'src-block
               (lambda (block)
                 (let ((name (org-element-property :name block)))
                   (when (and name (string-match-p infix name))
                     block))))))
        (dolist (block blocks)
          (goto-char (org-element-property :begin block))
          (org-babel-execute-src-block)))))

  ;; Emacs looks for "Local variables:" after the last "?\n?\f".
  (add-to-list 'safe-local-eval-forms
               '(apply 'my-org-eval-blocks-named '("emacs-lisp-setup")))
  (add-to-list 'safe-local-eval-forms
               '(apply 'my-org-eval-blocks-named '("python-setup")))

Easy LaTeX preamble editing

There are at least two ways (new and old) to edit the LateX preamble latex_header and latex-extra_header export options easily in LaTeX source or export blocks. This Org (info) file uses the new way, but keeps the old way for backwards compatibility.

The new way exploiting an idea of Omar Antolin Camarena is to code new <LANGUAGE>-modes allowing to edit in LaTeX mode and to export to LaTeX code with correct LaTeX preamble export setting prefixes. Here, are links to three posts exposing his idea:

  1. Export LaTeX macros to LaTeX and HTML/MathJax preambles (reddit),
  2. Export JavaScript source blocks to script tags in HTML (reddit),
  3. Export JavaScript source blocks to script tags in HTML (SX).

Listing lst:emacs-lisp-setup-latex-header implements this way by means of two new <LANGUAGE>-modes: latex-header and latex-extra-header.

  (with-eval-after-load 'org-src
    (defun prefix-all-lines (prefix body)
      (with-temp-buffer
        (insert body)
        (string-insert-rectangle (point-min) (point-max) prefix)
        (buffer-string)))

    (add-to-list 'org-src-lang-modes '("latex-header" . latex))

    (defvar org-babel-default-header-args:latex-header
      '((:exports . "results") (:results . "raw")))

    (defun org-babel-execute:latex-header (body _params)
      "Execute a block of LaTeX preamble lines with org-babel.
  This function is called by `org-babel-execute-src-block' and
  prefixes all lines with \"#+latex_header: \"."
      (prefix-all-lines "#+latex_header: " body))

    (add-to-list 'org-src-lang-modes '("latex-extra-header" . latex))

    (defvar org-babel-default-header-args:latex-extra-header
      '((:exports . "results") (:results . "raw")))

    (defun org-babel-execute:latex-extra-header (body _params)
      "Execute a block of LaTeX preamble lines with org-babel.
  This function is called by `org-babel-execute-src-block' and
  prefixes all lines with \"#+latex_extra_header: \"."
      (prefix-all-lines "#+latex_extra_header: " body)))

The old way is to use a special export attribute as in the function org-latex-header-blocks-filter in ox-extra.el. Apparently, nobody is using this broken function (broken, since it relies on support only in org-mode before 2014-11-11). Listing lst:org-latex-header-blocks-filter proposes a fix for org-latex-header-blocks-filter.

  (with-eval-after-load 'ox
    (defun org-latex-header-blocks-filter (backend)
      "Convert marked LaTeX export blocks to \"#+latex_header: \" lines.
  The marker is a line \"#+header: :header yes\" preceding the block.

  For instance, the LaTeX export block

  ,#+header: :header yes
  ,#+begin_export latex
  % This line converts to a LaTeX header line.
  ,#+end_export

  converts to

  \"#+latex_header: % This line converts to a LaTeX header line.\"."
      (when (org-export-derived-backend-p backend 'latex)
        (let ((blocks
               (org-element-map
                   (org-element-parse-buffer 'greater-element nil) 'export-block
                 (lambda (block)
                   (let ((type (org-element-property :type block))
                         (header (org-export-read-attribute :header block :header)))
                     (when (and (string= type "LATEX") (string= header "yes"))
                       block))))))
          (mapc (lambda (block)
                  ;; Set point to where to insert LaTeX header lines
                  ;; after deleting the block.
                  (goto-char (org-element-property :post-affiliated block))
                  (let ((lines
                         (split-string (org-element-property :value block) "\n")))
                    (delete-region (org-element-property :begin block)
                                   (org-element-property :end block))
                    (dolist (line lines)
                      (insert (concat "#+latex_header: "
                                      (replace-regexp-in-string "\\` *" "" line)
                                      "\n")))))
                ;; Reverse to go upwards to avoid wrecking the list of
                ;; block positions in the file that would occur in case
                ;; of going downwards.
                (reverse blocks))))))

This file uses the new way, while keeping the old way for backwards compatibility, because the new way feels less hackish than the old way. A practical difference is that new way source blocks (contrary to old way export blocks) do not work in #+SETUPFILE: <FILE>, but only in #+INCLUDE: <FILE> files.

#+SETUPFILE: and #+INCLUDE: usage

Evaluation of the source block in listing lst:make-source-block-with-export-keyword-settings produces the source block that exports to listing lst:source-file-export-keyword-settings which shows the first six lines of this /gav451/emacs.d/src/commit/fe9e04c09121f28c77f4c2bbde21f57b12ef81f8/README.org file. The last two lines show that setup-include.org is the argument for #+SETUPFILE: and #+INCLUDE:.

  echo "#+caption[Source file export keyword settings]:"
  echo "#+caption: The first six lines of README.org containing the export"
  echo "#+caption: keyword settings."
  echo "#+name: lst:source-file-export-keyword-settings"
  echo "#+begin_src org :tangle no"
  head -n 6 README.org
  echo -n "#+end_src"
#+title: Emacs setup for use with LaTeX, Org, and Python
#+author: Gerard Vermeulen
#+latex_class: article
#+latex_class_options: [11pt,a4paper,english,svgnames,tables]
#+setupfile: "setup-include.org"
#+include: "setup-include.org"

Listing lst:setup-include-export-keyword-settings tangles into the setup-include.org file and listing lst:by-backend-kbd-org-macro defines the tools for the Org mode kbd macro in setup-include.org.

  #+babel: :cache no
  #+macro: kbd (eval (by-backend-kbd-org-macro $1))
  #+property: header-args:emacs-lisp :exports code :results silent :tangle init.el
  #+property: header-args:org :tangle setup-include.org
  #+startup: content
  (with-eval-after-load 'ox
    (autoload 'htmlize-protect-string "htmlize" nil t)

    (defmacro by-backend (&rest body)
      `(cl-case org-export-current-backend ,@body))

    (defun by-backend-kbd-org-macro (keys)
      (by-backend
       (html (format "@@html:<kbd>%s</kbd>@@" (htmlize-protect-string keys)))
       (latex (format "@@latex:\\colorbox{PowderBlue}{\\texttt{%s}}@@" keys)))))

Listing lst:use-latex-header-1, lst:use-latex-header-2, lst:use-latex-header-3, and lst:use-latex-header-4 tangle into the setup-include.org file in order to create the LaTeX preamble.

  #+begin_src latex-header

    % LANGUAGE:
    \usepackage{babel}
    \usepackage{fvextra}
    \usepackage{csquotes}

    % LISTS:
    \usepackage{enumitem}
    \setlist{noitemsep}

    % LISTINGS:
    % Section 2.6 of caption-eng.pdf (texdoc caption) explains that the sign
    % of "skip" depends on the assumption "position=above" or "position=below".
    % The assumption should match the real caption position in the LaTeX code.
    \usepackage{caption}
    \usepackage[newfloat]{minted}
    \captionsetup[listing]{position=below,skip=0em}
    \usemintedstyle{xcode}

    % TABLES:
    % https://tex.stackexchange.com/questions/341205/
    % what-is-the-difference-between-tabular-tabular-and-tabularx-environments
    % https://emacs.stackexchange.com/questions/26179/
    % change-org-mode-table-style-just-for-latex-export
    % https://tex.stackexchange.com/questions/468585/
    % table-formatting-using-siunitx
    \usepackage{booktabs}
    \usepackage{colortbl}
    \usepackage{tabularx}  % DANGER: beware of Org table :width and :align options!

  #+end_src
  #+begin_src latex-header
    % PAGE LAYOUT:
    \usepackage{fancyhdr}
    \usepackage{lastpage}
    \usepackage[
      headheight=20mm,
      top=40mm,
      bottom=20mm,
      left=60pt,
      right=60pt,
      heightrounded,
      verbose,
    ]{geometry}

    % TECHNICS:
    \usepackage{siunitx}
    \usepackage{tikz}

  #+end_src
  #+begin_src latex-header
    % FLOAT BARRIERS:
    % https://tex.stackexchange.com/questions/118662/use-placeins-for-subsections
    % Make section an implicit float barrier:
    \usepackage[section]{placeins}
    % Make subsection an implicit float barrier:
    \makeatletter
    \AtBeginDocument{%
      \expandafter\renewcommand\expandafter\subsection\expandafter{%
        \expandafter\@fb@secFB\subsection
      }%
    }
    \makeatother
    % Make subsubsection an implicit float barrier:
    \makeatletter
    \AtBeginDocument{%
      \expandafter\renewcommand\expandafter\subsubsection\expandafter{%
        \expandafter\@fb@secFB\subsubsection
      }%
    }
    \makeatother

  #+end_src
  #+begin_src latex-header
    % FANCY HEADERS AND FOOTERS:
    % Add fancy headers and footers to normal pages.
    \pagestyle{fancy}
    \fancyhf{}
    \renewcommand{\footrulewidth}{0.4pt}
    \fancyfoot[C]{\emph{
        Emacs setup for use with \LaTeX{}, Org, and Python -- Gerard Vermeulen}}
    \renewcommand{\headrulewidth}{0.4pt}
    \fancyhead[L]{\includegraphics[height=1.8cm]{Org-mode-unicorn.png}}
    \fancyhead[C]{
      Page: \thepage/\pageref{LastPage} \\
      \text{ } \\
      \text{ } \\
      DRAFT
    }
    \fancyhead[R]{\includegraphics[height=1.8cm]{Emacs-logo.png}}

    % Add fancy header and footer to custom titlepage.
    % https://tex.stackexchange.com/questions/506102/
    % adding-header-and-footer-to-custom-titlepage
    \fancypagestyle{titlepage}{%
      \fancyhf{}
      \renewcommand{\footrulewidth}{0.4pt}
      \fancyfoot[C]{\emph{
          Emacs setup for use with \LaTeX{}, Org, and Python -- Gerard Vermeulen}}
      \renewcommand{\headrulewidth}{0.4pt}
      \fancyhead[L]{\includegraphics[height=1.8cm]{Org-mode-unicorn.png}}
      \fancyhead[C]{
        \pageref{LastPage} pages \\
        \text{ } \\
        \text{ } \\
        DRAFT
      }
      \fancyhead[R]{\includegraphics[height=1.8cm]{Emacs-logo.png}}
    }
    % #+latex_header: END.
  #+end_src

Advanced LaTeX export settings

How to customize org-latex-title-command only in this buffer

  (defun my-ox-latex-export-buffer-local-variables (title-page)
    (with-eval-after-load 'ox
      (make-variable-buffer-local 'org-export-before-parsing-hook)
      (cl-pushnew #'org-latex-header-blocks-filter
                  org-export-before-parsing-hook))
    (when (require 'ox-latex nil 'noerror)
      (make-variable-buffer-local 'org-latex-classes)
      (cl-pushnew '("article-local"
                    "\\documentclass[11pt]{article}
  [NO-DEFAULT-PACKAGES]
  [PACKAGES]
  [EXTRA]"
                    ("\\section{%s}" . "\\section*{%s}")
                    ("\\subsection{%s}" . "\\subsection*{%s}")
                    ("\\subsubsection{%s}" . "\\subsubsection*{%s}")
                    ("\\paragraph{%s}" . "\\paragraph*{%s}")
                    ("\\subparagraph{%s}" . "\\subparagraph*{%s}"))
                  org-latex-classes :key #'car :test #'equal)

      (make-variable-buffer-local 'org-latex-title-command)
      (setq org-latex-title-command (concat title-page))

      (make-variable-buffer-local 'org-latex-toc-command)
      (setq org-latex-toc-command
            (mapconcat 'identity '(""
                                   "\\tableofcontents\\label{toc}"
                                   "\\listoffigures"
                                   "\\listoflistings"
                                   "\\listoftables"
                                   "\\newpage"
                                   "") "\n"))

      (make-variable-buffer-local 'org-latex-subtitle-format)
      (setq org-latex-subtitle-format "")))

Listing lst:emacs-lisp-setup-call initializes the buffer local variables org-export-before-parsing-hook, org-latex-classes, org-latex-title-command, org-latex-toc-command, and org-latex-subtitle-format.

  (my-ox-latex-export-buffer-local-variables title-page)
  \begin{titlepage}
    %% https://tex.stackexchange.com/questions/506102/
    %% adding-header-and-footer-to-custom-titlepage
    \thispagestyle{titlepage}
    \begin{center}
      %% Title
      \begin{Huge}
        {\bf %t} \\
        \vspace{1em}
      \end{Huge}
      %% Author
      \begin{Large}
        {\bf %a} \\
        \vspace{1em}
      \end{Large}
    \end{center}
  \end{titlepage}

Worg: backend dependent execution

  #+latex_header: \usepackage{tikz}

  #+header: :file (by-backend (html "tree.svg") (latex nil))
  #+header: :results (by-backend (html "raw file") (latex "latex replace"))
  #+header: :headers '("\\usepackage{tikz}\n")
  #+begin_src latex
    \usetikzlibrary{trees}
    \begin{tikzpicture}
      \node [circle, draw, fill=red!20] at (0,0) {1}
      child { node [circle, draw, fill=blue!30] {2}
        child { node [circle, draw, fill=green!30] {3} }
        child { node [circle, draw, fill=yellow!30] {4} }};
    \end{tikzpicture}
  #+end_src

  # Setup
  #+begin_src emacs-lisp :exports none
    (with-eval-after-load 'ox
      (setq org-babel-latex-htlatex "htlatex")
      (defmacro by-backend (&rest body)
        `(cl-case org-export-current-backend ,@body)))
  #+end_src

This section updates the outdated Worg: backend dependent execution example to Emacs-27.2 and Org-9.5.2. It shows how to export PGF/TikZ images to:

  1. PDF by embedding in LaTeX.
  2. HTML by passing from LaTeX by PDF and to SVG.

Listing lst:backend-dependent-execution-update tangles to worg-backend-dependent-execution-update.org as either a standalone example or an example for inclusion. Finally, inclusion of worg-backend-dependent-execution-update.org produces here the figure with the nodes 1, 2, 3, and 4:\\

How to include such figures as floats remains an open question.

Programming

Emacs-lisp programming

Python programming

The Python Programming in Emacs wiki page lists options to enhance Emacs's built-in python-mode. Here, the focus is on two packages:

  1. Eglot - Emacs polyGLOT: an Emacs LSP client that stays out of your way. The maintainer also contributes to Emacs itself and has a deep understanding of the Way of Emacs. He refuses to add new features without seeing how they fit into the Way of Emacs as this discussion on org-mode source code blocks shows.
  2. Anaconda - code navigation, documentation lookup, and completion for Python.

In my opinion, eglot has more potential than anaconda, but anaconda is compatible with source code block editing while eglot is not. Listing lst:configure-python configures Python.

Python-mode

  (with-eval-after-load 'python
    (custom-set-variables
     '(python-indent-guess-indent-offset nil)
     '(python-shell-interpreter-args "-i -E")))

Pyenv

Listing lst:enable-pyenv-mode configures and enables pyenv-mode.

  (when (and (executable-find "pyenv")
             (require 'pyenv-mode nil 'noerror))
    (pyenv-mode +1)
    (pyenv-mode-set "3.9.9/envs/python-3.9.9"))

Eglot

Listing lst:configure-eglot+python-lsp-server-for-python (tangles to user-init-file) and lst:configure-eglot+jedi-language-server-for-python (does not tangle to user-init-file) configure eglot for Python using the python-lsp-server (or experimentally the less capable jedi-language-server). In order to enable all builtin python-lsp-server capabilities, ensure installation of the Python packages autopep8, flake8, pydocstyle, pylint, rope, and yapf. Listing lst:on-hack-local-variables-hook-eglot-maybe defines a hook function to launch eglot in presence of a proper .dir-locals.el file in the root directory of any Python project. Listing lst:eglot-directory-variables-for-python shows such a proper .dir-locals.el file.

  (with-eval-after-load 'eglot
    ;; (setq eglot-server-programs '((python-mode "pylsp")))
    (add-to-list 'eglot-server-programs '(python-mode "pylsp"))

    (setq-default
     eglot-workspace-configuration
     '((:pylsp . (:plugins (:jedi_completion (:cache_for ["astropy"]))))
       (:pylsp . (:plugins (:jedi (:auto_import_modules ["numpy"]))))
       (:pylsp . (:configurationSources ["flake8"])))))
  (with-eval-after-load 'eglot
    ;; (setq eglot-server-programs '((python-mode "jedi-language-server")))
    (add-to-list 'eglot-server-programs '(python-mode "jedi-language-server"))

    (setq-default
     eglot-workspace-configuration
     '((:jedi-language-server
        . (:plugins (:jedi (:jediSettings (:autoImportModules ["numpy"]))))))))
  (when (fboundp 'eglot-ensure)
    ;; The two hooks `after-change-major-mode-hook' and
    ;; `hack-local-variables-hook' are OK, but language mode hooks like
    ;; `python-mode-hook' are not.
    (add-hook 'hack-local-variables-hook
              (defun on-hack-local-variables-hook-eglot-maybe ()
                (when (and (derived-mode-p 'python-mode)
                           (assoc 'eglot-workspace-configuration
                                  dir-local-variables-alist))
                  (eglot-ensure)))))
  ;; Proposal for a .dir-locals.el file in the root of any Python project.
  ((python-mode
    . ((eglot-workspace-configuration
        . ((:pylsp . (:plugins (:jedi (:auto_import_modules ["numpy"]))))
           (:pylsp . (:plugins (:jedi_completion (:cache_for ["astropy"]))))
           (:pylsp . (:configurationSources ["flake8"])))))))

Jedi provides grammar checking and completion candidates to python-lsp-server. Only jedi-0.18.1 works with for instance numpy-1.22.0 in the sense that it handles universal functions provided that jedi does not parse but imports numpy-1.22.0 (see jedi issue #1744, #1745, and #1746).

Listing lst:make-pylsp-server-patch generates listing lst:show-pylsp-server-patch that shows the patch to make jedi import numpy-1.22.0 in order to serve the information to python-lsp-server allowing it to handle universal functions. Listing lst:show-pylsp-server-patch tangles to pylsp-auto-import-modules.patch.

  echo "#+attr_latex: :options breaklines"
  echo "#+caption[Show and tangle =pylsp-auto-import-modules.patch=]:"
  echo "#+caption: Show and tangle =pylsp-auto-import-modules.patch=."
  echo "#+name: lst:show-pylsp-server-patch"
  echo "#+begin_src diff :exports code :tangle pylsp-auto-import-modules.patch"
  git -C $HOME/VCS/python-lsp-server diff
  echo "#+end_src"
diff --git a/pylsp/config/schema.json b/pylsp/config/schema.json
index c29d78b..4f30101 100644
--- a/pylsp/config/schema.json
+++ b/pylsp/config/schema.json
@@ -69,6 +69,14 @@
       "default": null,
       "description": "List of errors and warnings to enable."
     },
+    "pylsp.plugins.jedi.auto_import_modules": {
+      "type": "array",
+        "default": ["numpy", "gi"],
+      "items": {
+        "type": "string"
+      },
+      "description": "List of module names for jedi to import (jedi.settings.auto_import_modules)."
+    },
     "pylsp.plugins.jedi.extra_paths": {
       "type": "array",
       "default": [],
diff --git a/pylsp/workspace.py b/pylsp/workspace.py
index bf312f6..4758b53 100644
--- a/pylsp/workspace.py
+++ b/pylsp/workspace.py
@@ -14,6 +14,8 @@ from . import lsp, uris, _utils

 log = logging.getLogger(__name__)

+DEFAULT_AUTO_IMPORT_MODULES = ["numpy", "gi"]
+
 # TODO: this is not the best e.g. we capture numbers
 RE_START_WORD = re.compile('[A-Za-z_0-9]*$')
 RE_END_WORD = re.compile('^[A-Za-z_0-9]*')
@@ -252,6 +254,8 @@ class Document:

         if self._config:
             jedi_settings = self._config.plugin_settings('jedi', document_path=self.path)
+            jedi.settings.auto_import_modules = jedi_settings.get('auto_import_modules',
+                                                                  DEFAULT_AUTO_IMPORT_MODULES)
             environment_path = jedi_settings.get('environment')
             extra_paths = jedi_settings.get('extra_paths') or []
             env_vars = jedi_settings.get('env_vars')

Finally, here are a few links covering how to integrate Emacs, Python and a Python LSP server:

  1. Building Your Own Emacs IDE with LSP
  2. Doom Emacs and Language Servers
  3. Eglot based Emacs Python IDE
  4. Getting started with lsp-mode for Python
  5. Python & Emacs, Take 3
  6. Python with Emacs: py(v)env and lsp-mode

Anaconda

Listing lst:configure-anaconda+company-for-python and lst:define-my-toggle-anaconda-mode configure anaconda. See elpy-module-company for how to handle company-backends as a local variable in listing lst:configure-anaconda+company-for-python. The call to advice-add in listing lst:define-my-toggle-anaconda-mode opens Python org-edit-src-code buffers in anaconda-mode.

  (with-eval-after-load 'python
    (with-eval-after-load 'company
      (when (and (fboundp 'anaconda-mode)
                 (fboundp 'company-anaconda))
        (defun my-disable-anaconda-mode ()
          (when (derived-mode-p 'python-mode)
            (anaconda-mode -1)
            (make-variable-buffer-local 'company-backends)
            (setq company-backends
                  (delq 'company-anaconda
                        (mapcar #'identity company-backends)))
            (anaconda-eldoc-mode -1)))
        (defun my-enable-anaconda-mode ()
          (when (derived-mode-p 'python-mode)
            (anaconda-mode +1)
            (make-variable-buffer-local 'company-backends)
            (setq company-backends
                  (cons 'company-anaconda
                        (delq 'company-semantic
                              (delq 'company-capf
                                    (mapcar #'identity company-backends)))))
            (anaconda-eldoc-mode
             (if (file-remote-p default-directory) -1 1)))))))
  (with-eval-after-load 'python
    (unless (and (fboundp 'my-disable-anaconda-mode)
                 (fboundp 'my-enable-anaconda-mode))
      (when (fboundp 'anaconda-mode)
        (defun my-disable-anaconda-mode ()
          (when (derived-mode-p 'python-mode)
            (anaconda-mode -1)
            (anaconda-eldoc-mode -1)))
        (defun my-enable-anaconda-mode ()
          (when (derived-mode-p 'python-mode)
            (anaconda-mode +1)
            (anaconda-eldoc-mode
             (if (file-remote-p default-directory) -1 1))))))

    (when (fboundp 'my-enable-anaconda-mode)
      (advice-add 'org-edit-src-code :after #'my-enable-anaconda-mode))
    (when (and (fboundp 'my-disable-anaconda-mode)
               (fboundp 'my-enable-anaconda-mode))
      (defun my-toggle-anaconda-mode ()
        "Toggle anaconda-mode with bells and whistles."
        (interactive)
        (if (bound-and-true-p anaconda-mode)
            (my-disable-anaconda-mode)
          (my-enable-anaconda-mode)))))

Jedi

Listing lst:example-py is a Python example to test whether jedi in combination with and either anaconda or eglot works when coding certain functions of for instance numpy and scipy.

  import numpy
  import astropy.units as apu

  a = numpy.arange(0, 11)
  a = numpy.linspace(0, 10, num=11)
  a = numpy.arccos(a)
  q = apu.Quantity(a, apu.meter)
  print(q)

Editing

Enable disabled commands and inform

Execute src_emacs-lisp{(find-library "novice")} to see how Emacs prevents new users from shooting themselves in the feet.

  (setq disabled-command-function
        (defun my-enable-this-command (&rest _args)
          "Called when a disabled command is executed.
  Enable it and re-execute it."
          (put this-command 'disabled nil)
          (message "You typed %s.  %s was disabled until now."
                   (key-description (this-command-keys)) this-command)
          (sit-for 0)
          (call-interactively this-command)))

Narrowing

Narrowing means focusing in on some portion of the buffer and widening means focussing out on the whole buffer. This allows to concentrate temporarily on for instance a particular function or paragraph by removing clutter. The "Do What I Mean" narrow-or-widen-dwim function allows to toggle between narrowed and widened buffer states. Here, the function narrow-or-widen-dwim operates also on tables by means of org-narrow-to-table.

  (defun org-narrow-to-table ()
    "Narrow buffer to current table."
    (interactive)
    (if (org-table-p)
        (narrow-to-region (org-table-begin) (org-table-end))
      (user-error "Not in a table")))

  (defun narrow-or-widen-dwim (p)
    "Widen if buffer is narrowed, narrow-dwim otherwise.
  Dwim means: region, org-src-block, org-subtree, or defun,
  whichever applies first. Narrowing to org-src-block actually
  calls `org-edit-src-code'.
  With prefix P, don't widen, just narrow even if buffer is
  already narrowed."
    (interactive "P")
    (declare (interactive-only))
    (cond ((and (buffer-narrowed-p) (not p)) (widen))
          ((and (bound-and-true-p org-src-mode) (not p))
           (org-edit-src-exit))
          ((region-active-p)
           (narrow-to-region (region-beginning) (region-end)))
          ((derived-mode-p 'org-mode)
           (or (ignore-errors (org-edit-src-code))
               (ignore-errors (org-narrow-to-block))
               (ignore-errors (org-narrow-to-table))
               (org-narrow-to-subtree)))
          ((derived-mode-p 'latex-mode)
           (LaTeX-narrow-to-environment))
          ((derived-mode-p 'tex-mode)
           (TeX-narrow-to-group))
          (t (narrow-to-defun))))

  (define-key ctl-x-map (kbd "n t") #'org-narrow-to-table)
  (define-key ctl-x-map (kbd "C-n") #'narrow-or-widen-dwim)

Synchronal multiple-region editing

  (unless noninteractive
    (require 'iedit nil 'noerror))

Unobtrusive whitespace trimming

  (unless noninteractive
    (when (require 'ws-butler nil 'noerror)
      (custom-set-variables
       '(ws-butler-keep-whitespace-before-point nil))
      (add-hook 'prog-mode-hook #'ws-butler-mode)
      (add-hook 'text-mode-hook #'ws-butler-mode)))

Structural editing

Structural editing keeps character pairs (for instance parentheses, curly and square brackets as well as single and double quotes) balanced to leave code (for instance Lisp and Python) and text (for instance LaTeX and Org) structure intact. I use smartparens which offers a normal mode (smartparens-mode) and a a strict mode (smartparens-strict-mode). Although both modes insert character pairs, the normal mode allows to delete one of the paired characters easily while the strict mode does not. Therefore, the strict mode is more for code editing since it never breaks programming language rules and the normal mode is more for text editing where structure is a matter of convention instead of programming language rules.

For instance, the strict mode in Python allows to delete entire lists, tuples, or the arguments after the cursor (what Emacs calls point) in a function call without breaking the character pair balance. In order to repair a broken character pair balance, insert a single character by prefixing it with "C-q" bound to quoted-insert.

The smartparens documentation targets experienced Emacs users. The following links show how to put the documentation to practical use:

  1. Omar Antolin's gist "my-smartparens-config.el" is the first place to look for how to tweak smartparens. However, the gist may be partially obsolete, since it is not part of his current Emacs configuration.
  2. How to enable smartparens in the minibuffer after eval-expression explains how the machinery after the first and after later usages of eval-expression differ and discusses options how to handle those differences.

Listing lst:configure-smartparens aims to configure smartparens for Elisp, LaTeX, Org, and Python.

  (unless noninteractive
    ;; To disables pairing of the quote character for lisp modes,
    ;; require smartparens-config instead of smartparens.
    (when (require 'smartparens-config nil 'noerror)
      (custom-set-variables
       '(sp-base-key-bindings 'sp)
       '(sp-override-key-bindings '(("C-(" . sp-backward-barf-sexp)
                                    ("C-)" . sp-forward-slurp-sexp))))

      (dolist (hook '(prog-mode-hook
                      text-mode-hook))
        (add-hook hook #'smartparens-mode))

      ;; Hook on the specific `eval-expression-minibuffer-setup-hook'
      ;; and not on the general `minibuffer-setup-hook'.
      (dolist (hook '(emacs-lisp-mode-hook
                      eval-expression-minibuffer-setup-hook
                      ielm-mode-hook
                      python-mode-hook))
        (add-hook hook #'smartparens-strict-mode))

      ;; Tweak for the call to `smartparens-strict-mode' hooked on
      ;; `eval-expression-minibuffer-setup-hook'.
      (sp-with-modes '(fundamental-mode           ; first usage.
                       minibuffer-inactive-mode)  ; later usage.
        (sp-local-pair "'" nil :actions nil))

      ;; https://xenodium.com/emacs-smartparens-auto-indent/index.html
      (defun indent-between-pair (&rest _ignored)
        (newline)
        (indent-according-to-mode)
        (forward-line -1)
        (indent-according-to-mode))

      (dolist (left '("(" "[" "{"))
        (sp-local-pair 'prog-mode left
                       nil :post-handlers '((indent-between-pair "RET"))))

      (show-smartparens-global-mode +1)))

Electric operators

Listing lst:configure-electric-operator-mode configures electric-operator-mode to add spaces around operators for compatibility with for instance the Black code formatter for Python.

  (when (fboundp 'electric-operator-mode)
    (add-hook 'c-mode-common #'electric-operator-mode)
    (add-hook 'python-mode-hook #'electric-operator-mode))

Smart snippets

  (when (require 'yasnippet nil 'noerror)
    (custom-set-variables
     '(yas-alias-to-yas/prefix-p nil))
    (yas-global-mode +1))

Appearance

This setup does not configure custom themes (info) in order to prevent endless useless tweaking.

Text faces (or styles)

See the note on mixed font heights in Emacs for how to setup fonts properly. It boils down to two rules:

  1. The height of the default face must be an integer number to make the height a physical quantity.
  2. The heights of all other faces must be real numbers to scale those heights with respect to the height of the face (those heights default to 1.0 for no scaling).

The code in listing lst:configure-face-attributes source implements those rules. In case of proper initialization of all face heigths, font scaling is easy as listing lst:my-set-default-face-height shows. Finally, the code in listing lst:my-invert-default-face allows swapping the foreground and background colors of the default face on all frames and the code in listing lst:custom-set-faces improves the visibility of some default faces.

  (unless noninteractive
    ;; Set face attributes.
    (cond
     ((eq system-type 'darwin)
      (set-face-attribute 'default nil :family "Hack" :height 120)
      (set-face-attribute 'fixed-pitch nil :family "Hack")
      (set-face-attribute 'variable-pitch nil :family "FiraGo"))
     ((eq system-type 'gnu/linux)
      (set-face-attribute 'default nil :family "Hack" :height 110)
      (set-face-attribute 'fixed-pitch nil :family "Hack")
      (set-face-attribute 'variable-pitch nil :family "FiraGo"))
     (t
      (set-face-attribute 'default nil :family "Hack" :height 110)
      (set-face-attribute 'fixed-pitch nil :family "Hack")
      (set-face-attribute 'variable-pitch nil :family "DejaVu Sans"))))
  (unless noninteractive
    (defun my-set-default-face-height ()
      "Set the default face height in all current and future frames.

  Scale all other faces with a height that is a real number."
      (interactive)
      (let* ((prompt (format "face heigth (%s): "
                             (face-attribute 'default :height)))
             (choices (mapcar #'number-to-string
                              (number-sequence 50 200 10)))
             (height (string-to-number
                      (completing-read prompt choices nil 'require-match))))
        (message "Setting the height of the default face to %s" height)
        (set-face-attribute 'default nil :height height))))
  (unless noninteractive
    (defun my-invert-default-face ()
      "Invert the default face."
      (interactive)
      (invert-face 'default)))
  (with-eval-after-load 'faces
    (custom-set-faces
     '(region ((((class color) (min-colors 88) (background light) (type gtk))
                :background "lightgoldenrod2")))))

Visualize color codes and names

Listing lst:enable-rainbow-mode enables rainbow-mode to colorize color codes and names in buffers for debugging.

  (when (fboundp 'rainbow-mode)
    (custom-set-variables
     '(rainbow-x-colors-major-mode-list
       '(c++-mode
         c-mode
         emacs-lisp-mode
         inferior-emacs-lisp-mode
         java-mode
         lisp-interaction-mode
         org-mode
         python-mode)))
    (rainbow-mode +1))

Flash the line around point for visual feedback

  (unless noninteractive
    ;; https://karthinks.com/software/batteries-included-with-emacs/
    ;; https://www.reddit.com/r/emacs/comments/jwhr6g/batteries_included_with_emacs/
    (defun my-pulse-one-line (&rest _)
      "Pulse the current line."
      (let ((pulse-iterations 16)
            (pulse-delay 0.1))
        (pulse-momentary-highlight-one-line (point))))
    (dolist (command '(scroll-up-command
                       scroll-down-command
                       recenter-top-bottom
                       other-window))
      (advice-add command :after #'my-pulse-one-line)))

Applications

Elfeed: Emacs web feed reader

  (autoload 'elfeed "elfeed" nil t)
  (global-set-key (kbd "C-x w") #'elfeed)

  (with-eval-after-load 'elfeed
    (custom-set-variables
     '(elfeed-feeds
       '(("http://www.howardism.org/index.xml" h-abrams)
         ("https://ambrevar.xyz/atom.xml" p-neirhardt)
         ("https://emacshorrors.com/feed.atom" v-schneidermann)
         ("https://emacsninja.com/emacs.atom" v-schneidermann)
         ("https://feeds.feedburner.com/InterceptedWithJeremyScahill" j-scahill)
         ("https://nullprogram.com/feed/" c-wellons)
         ("https://oremacs.com/atom.xml" o-krehel)
         ("https://planet.emacslife.com/atom.xml" planet-emacs)
         ("https://protesilaos.com/codelog.xml" p-stavrou)
         ("https://sachachua.com/blog/category/emacs/feed" s-chua)
         ("https://sciencescitoyennes.org/feed/" sciences)
         ("https://updates.orgmode.org/feed/updates" org-updates)
         ("https://www.aclu.org/taxonomy/feed-term/2152/feed" aclu)
         ("https://www.bof.nl/rss/" bof)
         ("https://www.democracynow.org/podcast-video.xml" dn)
         ("https://www.laquadrature.net/fr/rss.xml" lqdn)
         ("https://www.lemonde.fr/blog/huet/feed/" sciences)))))

Emacs Multimedia System (info)

  (custom-set-variables
   '(emms-mode-line-format "")
   '(emms-player-list '(emms-player-mpd emms-player-mpv))
   `(emms-player-mpd-music-directory ,(expand-file-name "~/Music"))
   '(emms-player-mpd-server-name "localhost")
   '(emms-player-mpd-server-port "6600")
   '(emms-player-mpd-verbose t)
   '(emms-playing-time-display-format " %s ")
   '(emms-playlist-mode-center-when-go t))

  (defun my-emms-print-metadata-find ()
    (require 'find-func)
    (locate-file
     "emms-print-metadata"
     (expand-file-name
      "src"
      (file-name-directory (find-library-name "emms")))
     exec-suffixes #'file-executable-p))

  (with-eval-after-load 'emms
    (require 'emms-info-libtag)
    (let ((emms-print-metadata (my-emms-print-metadata-find)))
      (when emms-print-metadata
        (custom-set-variables
         '(emms-info-functions nil)
         `(emms-info-libtag-program-name ,emms-print-metadata))
        (add-hook 'emms-info-functions #'emms-info-libtag))))

  (with-eval-after-load 'elfeed-show
    (when (require 'emms-setup nil 'noerror)
      (emms-all)))

  (autoload 'emms-streams "emms-streams" nil 'interactive)
  (with-eval-after-load 'emms-streams (emms-all))

Local variables linking to Latexmk save-compile-display-loop

Only the Org source file shows the local variables footer.