Vim, virtual environments, validation

Introduction

Vim and Neovim both ship with embedded terminal emulators now, which is great news if you’re like me and want to do everything in Vim that you can. Tools like Projectionist and fzf make it as easy to manage web servers, builds, tests, and scratch terminals as it is to manage regular text buffers. There are plenty of articles about this work flow already (search “Neovim as as tmux replacement”), but I want to highlight a chunk of configuration I have for dealing with Python’s virtual environments.

Getting linting and completion tools to work with multiple virtual environments in a single Vim instance is tricky. In the tried-and-tested Vim + tmux setup this is easy to handle, as you simply launch a separate Vim instance for each project into the corresponding virtual environment. Each instance launches with the correct environment variables to make your tools work for that project (and that project only).

Why bother doing it any differently? By piling all of your buffers into a single Vim instance, you don’t have to think about navigating through tmux/your terminal emulator/your window manager any more. There’s no more friction when switching between unrelated buffers in different projects than there is switching between two that came out of the same folder. :b partialbuffername<CR> will bring up whatever you’re looking for right away, or you can use a fuzzy finder if you prefer something more visual. The improvement in efficiency becomes obvious quickly when you routinely work on many buffers scattered among many projects.

Setup

I’ll start with the configuration so you can get an idea of how it all fits together. Your VCS, linting plugin, linting tools, completion plugin, project structure and environment management are all variables here so take it with a large pinch of salt.

function! PythonVenvExecutable()
  " Give up if we're not looking at a Python buffer
  if &filetype !=# 'python'
    return
  endif

  " Try to get Projectionist root directory name
  let l:venv_name = fnamemodify(projectionist#path(), ':t')

  " Fall back to Git repo name
  if l:venv_name ==# ''
    let l:venv_name = fnamemodify(fugitive#extract_git_dir(@%), ':h:t')
  endif

  " Default to global Python/pylint
  let l:python_binary = '/usr/bin/python'
  let l:pylint_binary = '/usr/local/bin/pylint'

  " Use the virtual environment Python/pylint if it exists
  let l:venv_root = $WORKON_HOME.'/'.l:venv_name
  if l:venv_name !=# '.' && isdirectory(l:venv_root)
    let l:python_binary = l:venv_root.'/bin/python'
    let l:pylint_binary = l:venv_root.'/bin/pylint'
  endif

  " Configure ALE
  let b:ale_python_pylint_executable = l:pylint_binary

  " Configure Completor
  if exists('g:completor_python_binary')
    if l:python_binary !=# g:completor_python_binary
      " If we've changed environments
      let g:completor_python_binary = l:python_binary
      silent call completor#daemon#kill()
    endif
  else
    " If this is the first Python buffer in the session
    let g:completor_python_binary = l:python_binary
  endif
endfunction
autocmd BufWinEnter,WinEnter * :call PythonVenvExecutable()

To get completion and linting with virtual environment support, we first need to figure out how to follow a buffer’s path to the Python binaries in the corresponding virtual environment. I use virtualenvwrapper to manage my environments (but I’d love to hear from anyone using pipenv to do this!). To find the root directory of the project that a given buffer belongs to, I’m using Fugitive and Projectionist. Fugitive is a must-have if you use Git at all, and Projectionist is a project management tool that works brilliantly with this setup.

Once we’ve found the project directory, we need to find the right virtual environment. If you keep your environment directory in the project root you just need to append venv or such to it. If you use virtualenvwrapper you’ll have to append the directory name to the $WORKON_HOME environment variable.

Next, we have to configure our linting and completion plugins to use the given executables. I use Asynchronous Lint Engine with Pylint for linting and Completor for completion. Both have nice, simple configuration. Completor comes highly recommended for this setup since it exposes a function for re-crawling the Python module path, which is essential. This was added on my request by the author Wei Zhang, so many thanks to them. I’ll discuss an alternative solution later if this isn’t an option with your completion plugin of choice. Remember that you’ll need to have Pylint and Jedi installed in each environment.

Finally, we have to wrap the whole thing in an autocmd that executes when switching buffers or windows. Now your lint results and completion suggestions for Python buffers will be representative of their environment!

Alternatives

Completor is the only completion manager I’m aware of that exposes a function for re-caching your Python modules. If you prefer something else, there’s an alternative which I’ve used in the past. You can use envplus to make a new virtual environment that combines all of your other ones, install Jedi in it, then point your completion manager at it. This might also be of interest if you’re using an older machine and you don’t want it to crawl the Python path more than once.

The disadvantages are that it’s untested, unmaintained, and doesn’t have Python 3 support (I’ve opened a PR for that which you can clone and install). Also, Jedi will be anchored to whichever version of Python you’ve chosen for the environment, and the completions won’t be wholly accurate if your merged environments have multiple versions of the same package. It’s better than nothing though!

Update 10/02/18: I’ve taken to using vim-test and I’m happy to report that it works brilliantly with this setup and saves you having to write a Projectionist entry for the test runner in each of your projects. For example, add

let g:test#python#djangotest#executable = l:python_binary.' manage.py test'

to the autocmd for Django projects.