RSS Icon

Topics

View Comments

Turning Vim into a modern Python IDE

Topics: python vim

TL;DR:

$ git clone https://github.com/sontek/dotfiles.git
$ cd dotfiles
$ ./install.sh vim

Download PDF Version

Intro

Back in 2008, I wrote the article Python with a modular IDE (Vim). Years later, I have people e-mailing me and commenting daily asking for more information, even though most of the information in it is outdated. Here is the modern way to work with Python and Vim to achieve the perfect environment.

Because one of the most important parts about a development environment is the ability to easily reproduce across machines, we are going to store our vim configuration in git:

$ mkdir ~/.vim/
$ mkdir ~/.vim/{autoload,bundle}
$ cd ~/.vim/
$ git init

The purpose of the autoload directory is to automatically load the vim plugin Pathogen, which we'll then use to load all other plugins that are located in the bundle directory. So download pathogen and put it in your autoload folder.

You'll need to add the following to your ~/.vimrc so that pathogen will be loaded properly. Filetype detection must be off when you run the commands so its best to execute them first:

filetype off
call pathogen#runtime_append_all_bundles()
call pathogen#helptags()

Now lets add all of the vim plugins we plan on using as submodules to our git repository:

git submodule add http://github.com/tpope/vim-fugitive.git bundle/fugitive
git submodule add https://github.com/msanders/snipmate.vim.git bundle/snipmate
git submodule add https://github.com/tpope/vim-surround.git bundle/surround
git submodule add https://github.com/tpope/vim-git.git bundle/git
git submodule add https://github.com/ervandew/supertab.git bundle/supertab
git submodule add https://github.com/sontek/minibufexpl.vim.git bundle/minibufexpl
git submodule add https://github.com/wincent/Command-T.git bundle/command-t
git submodule add https://github.com/mitechie/pyflakes-pathogen.git
git submodule add https://github.com/mileszs/ack.vim.git bundle/ack
git submodule add https://github.com/sjl/gundo.vim.git bundle/gundo
git submodule add https://github.com/fs111/pydoc.vim.git bundle/pydoc
git submodule add https://github.com/vim-scripts/pep8.git bundle/pep8
git submodule add https://github.com/alfredodeza/pytest.vim.git bundle/py.test
git submodule add https://github.com/reinh/vim-makegreen bundle/makegreen
git submodule add https://github.com/vim-scripts/TaskList.vim.git bundle/tasklist
git submodule add https://github.com/vim-scripts/The-NERD-tree.git bundle/nerdtree
git submodule add https://github.com/sontek/rope-vim.git bundle/ropevim
git submodule init
git submodule update
git submodule foreach git submodule init
git submodule foreach git submodule update

Thats it! Now that we've got our vim configuration in git!

Now lets look at how to use each of these plugins to improve the power of vim:

Basic Editing and Debugging

Code Folding

Lets first enable code folding. This makes it a lot easier to organize your code and hide portions that you aren't interested in working on. This is quite easy for Python, since whitespace is required.

In your ~/.vimrc just add:

set foldmethod=indent
set foldlevel=99

Then you will be able to be inside a method and type 'za' to open and close a fold.

Window Splits

Sometimes code folding isn't enough; you may need to start opening up multiple windows and working on multiple files at once or different locations within the same file. To do this in vim, you can use these shortcuts:

Vertical Split : Ctrl+w + v
Horizontal Split: Ctrl+w + s
Close current windows: Ctrl+w + q

I also like to bind Ctrl+<movement> keys to move around the windows, instead of using Ctrl+w + <movement>:

map <c-j> <c-w>j
map <c-k> <c-w>k
map <c-l> <c-w>l
map <c-h> <c-w>h
http://i.imgur.com/krj0l.png

Snippets

The next tweak that really speeds up development is using snipmate. We've already included it in our bundle/ folder so its already enabled. Try opening up a python file and typing 'def<tab>'. It should stub out a method definition for you and allow you to tab through and fill out the arguments, doc string, etc.

I also like to create my own snippets folder to put in some custom snippets:

$ mkdir ~/.vim/snippets
$ vim ~/.vim/snippets/python.snippets

Put this in the file:

snippet pdb
    import pdb; pdb.set_trace()

Now you can type pdb<tab> and it'll insert your breakpoint!

Task lists

Another really useful thing is to mark some of your code as TODO or FIXME! I know we all like to think we write perfect code, but sometimes you just have to settle and leave a note for yourself to come back later. One of the plugins we included was the tasklist plugin that will allow us to search all open buffers for things to fix. Just add a mapping to open it in ~/.vimrc:

map <leader>td <Plug>TaskList

Now you can hit <leader>td to open your task list and hit 'q' to close it. You can also hit enter on the task to jump to the buffer and line that it is placed on.

Revision History

The final basic editing tweak I suggest everyone start utilizing is the Gundo plugin. It'll allow you to view diff's of every save on a file you've made and allow you to quickly revert back and forth:

http://i.imgur.com/2NrPS.png

Just bind a key in your .vimrc to toggle the Gundo window:

map <leader>g :GundoToggle<CR>

Syntax Highlighting and Validation

Simply enable syntax highlighting in your ~/.vimrc:

syntax on                           " syntax highlighing
filetype on                          " try to detect filetypes
filetype plugin indent on    " enable loading indent file for filetype

Because we enabled pyflakes when we added it as a submodule in ~/.vim/bundle, it will notify you about unused imports and invalid syntax. It will save you a lot of time saving and running just to find out you missed a colon. I like to tell it not use the quickfix window:

let g:pyflakes_use_quickfix = 0
http://i.imgur.com/ZfjFe.png

Pep8

The final plugin that really helps validate your code is the pep8 plugin, it'll make sure your code is consistent across all projects. Add a key mapping to your ~/.vimrc and then you'll be able to jump to each of the pep8 violations in the quickfix window:

let g:pep8_map='<leader>8'
http://i.imgur.com/VU9AB.png

Tab Completion and Documentation

Vim has many different code completion options. We are going to use the SuperTab plugin to check the context of the code you are working on and choose the best for the situation. We've already enabled the SuperTab plugin in the bundle/ folder, so we just have to configure it to be context sensitive and to enable omni code completion in your ~/.vimrc:

au FileType python set omnifunc=pythoncomplete#Complete
let g:SuperTabDefaultCompletionType = "context"

Now we just enable the menu and pydoc preview to get the most useful information out of the code completion:

set completeopt=menuone,longest,preview
http://i.imgur.com/g4lxP.png

We also enabled the pydoc plugin at the beginning with all the submodules; that gives us the ability to hit <leader>pw when our cursor is on a module and have a new window open with the whole documentation page for it.

Code Navigation

Buffers

The most important part about navigating code within vim, is to completely understand how to use buffers. There is no reason to use tabs. Open files with :e <filename> to place in a buffer. We already installed the minibufexpl plugin, so you will already visually see every buffer opened. You can also get a list of them doing :buffers.

You can switch between the buffers using b<number>, such as :b1 for the first buffer. You can also use its name to match, so you can type :b mod<tab> to autocomplete opening the models.py buffer. You need to make sure you are using the minibufexpl from my github since it has patches that make it much better to work with.

To close a buffer you use :bd or :bw.

File Browser

NERD Tree is a project file browser. I must admit I used this heavily back when I was migrating from Visual Studio and used to the Solution Explorer, but I rarely use it anymore. Command-T is usually all you'll need. It is useful when you are getting to know a new codebase for the first time though. Lets bind a shortcut key for opening it:

map <leader>n :NERDTreeToggle<CR>
http://i.imgur.com/R4ZzQ.png

Refactoring and Go to definition

Ropevim is also a great tool that will allow you to navigate around your code. It supports automatically inserting import statements, goto definition, refactoring, and code completion. You'll really want to read up on everything it does, but the two big things I use it for is to jump to function or class definitions quickly and to rename things (including all their references).

For instance, if you are using django and you place your cursor over the class models.Model you reference and then called :RopeGotoDefintion, it would jump you straight to the django library to that class definition. We already have it installed in our bundles, so we bind it to a key to use it:

map <leader>j :RopeGotoDefinition<CR>
map <leader>r :RopeRename<CR>

Searching

The final tool that really speeds up navigating your code is the Ack plugin. Ack is similar to grep, but much better in my opinion. You can fuzzy text search for anything in your code (variable name, class, method, etc) and it'll give you a list of files and line numbers where they are defined so you can quickly cycle through them. Just bind the searching to a key:

nmap <leader>a <Esc>:Ack!

We use ! at the end of it so it doesn't open the first result automatically.

Integration with Git

We installed 2 plugins, git.vim and fugitive, that give us all the integration we need. Git.vim will provide us syntax highlighting for git configuration files; fugitive provides a great interface for interacting with git including getting diffs, status updates, committing, and moving files.

Fugitive also allows you to view what branch you are working in directly from vim. Add this to your statusline in ~/.vimrc:

%{fugitive#statusline()}

The big commands you need to know:

  • Gblame: This allows you to view a line by line comparison of who the last person to touch that line of code is.
  • Gwrite: This will stage your file for commit, basically doing git add <filename>
  • Gread: This will basically run a git checkout <filename>
  • Gcommit: This will just run git commit. Since its in a vim buffer, you can use keyword completion (Ctrl-N), like test_all<Ctrl-N> to find the method name in your buffer and complete it for the commit message. You can also use + and - on the filenames in the message to stage/unstage them for the commit.
http://i.imgur.com/NuRRj.png

Test Integration

django nose

Test runner integration really depends on the testing library you are using and what type of tests you are running but we included a great generic plugin called MakeGreen that executes off of vim's makeprg variable. So for instance, if you are using django with django-nose you could define a shortcut key in your ~/.vimrc like this:

map <leader>dt :set makeprg=python\ manage.py\ test\|:call MakeGreen()<CR>

This will just give you a green bar at the bottom of vim if your test passed or a red bar with the message of the failed test if it doesn't. Very simple.

py.test

I also included the py.test vim plugin for those who prefer it. This plugin has a lot more functionality including executing individual tests by class, file, or method. You can also cycle through the individual assertion errors. I have the following bindings:

" Execute the tests
nmap <silent><Leader>tf <Esc>:Pytest file<CR>
nmap <silent><Leader>tc <Esc>:Pytest class<CR>
nmap <silent><Leader>tm <Esc>:Pytest method<CR>
" cycle through test errors
nmap <silent><Leader>tn <Esc>:Pytest next<CR>
nmap <silent><Leader>tp <Esc>:Pytest previous<CR>
nmap <silent><Leader>te <Esc>:Pytest error<CR>
http://i.imgur.com/RAE7v.png

Virtualenv

Vim doesn't realize that you are in a virtualenv so it wont give you code completion for libraries only installed there. Add the following script to your ~/.vimrc to fix it:

" Add the virtualenv's site-packages to vim path
py << EOF
import os.path
import sys
import vim
if 'VIRTUAL_ENV' in os.environ:
    project_base_dir = os.environ['VIRTUAL_ENV']
    sys.path.insert(0, project_base_dir)
    activate_this = os.path.join(project_base_dir, 'bin/activate_this.py')
    execfile(activate_this, dict(__file__=activate_this))
EOF

Django

The only true django tweak I make is before I open vim I'll export the DJANGO_SETTINGS_MODULE environment so that I get code completion for django modules as well:

export DJANGO_SETTINGS_MODULE=project.settings

Random Tips

If you want to find a new color scheme just go to http://code.google.com/p/vimcolorschemetest/ to preview a large selection.

© John Anderson <sontek@gmail.com> 2011

View Comments

Python with a modular IDE (Vim)

Topics: python vim
THIS IS OUT DATED... PLEASE VISIT THE UPDATED POST here!

On Thursday, May 9th, 2008 the Utah Python User Group decided to settle the debate that has plagued us developers since the beginning of time: If you were a programming language, what editor would you use?

I was tasked with showing Eclipse with the PyDev plugin in all its glory–but we all know–real men / developers don’t use IDE’s, so we are going to talk about using Python and Vim together, reaching a state of Zen that the Dalai LLama would be jealous of and establishing more Feng Shui than Martha Stewart’s Kitchen.

Freely jump between your code and python class libraries

There are 2 ways to add your ability to jump between python class libraries, the first is to setup vim to know where the Python libs are so you can use ‘gf’ to get to them (gf is goto file). You can do this by adding this snippet to your .vimrc:

python << EOF
import os
import sys
import vim
for p in sys.path:
    if os.path.isdir(p):
        vim.command(r"set path+=%s" % (p.replace(" ", r"\ ")))
EOF

With that snippet you will be able to go to your import statements and hit ‘gf’ on one of them and it’ll jump you to that file.

Continuing accessibility of the Python class libraries we are going to want to use ctags to generate an index of all the code for vim to reference:

$ ctags -R -f ~/.vim/tags/python.ctags /usr/lib/python2.5/

and then in your .vimrc

set tags+=$HOME/.vim/tags/python.ctags

This will give you the ability to use CTRL+] to jump to the method/property under your cursor in the system libraries and CTRL+T to jump back to your source code.

I also have 2 tweaks in my .vimrc so you can use CTRL+LeftArrow and CTRL+RightArrow to move between the files with more natural key bindings.

map <silent><C-Left> <C-T>
map <silent><C-Right> <C-]>

You can also see all the tags you’ve been to with “:tags”

Code Completion

To enable code completion support for Python in Vim you should be able to add the following line to your .vimrc:

autocmd FileType python set omnifunc=pythoncomplete#Complete

but this relies on the fact that your distro compiled python support into vim (which they should!).

Then all you have to do to use your code completion is hit the unnatural, wrist breaking, keystrokes CTRL+X, CTRL+O. I’ve re-bound the code completion to CTRL+Space since we are making vim an IDE! Add this command to your .vimrc to get the better keybinding:

inoremap <Nul> <C-x><C-o>

Documentation

No IDE is complete without the ability to access the class libraries documentation! You’ll need to grab this vim plugin. This gives you the ability to type :Pydoc os.path or use the keystrokes <Leader>pw and <Leader>pW to search for the item under the cursor. (Vim’s default <Leader> is “\”).

Syntax Checking

Vim already has built in syntax highlighting for python but I have a small tweak to vim to give you notifications of small syntax errors like forgetting a colon after a for loop. Create a file called ~/.vim/syntax/python.vim and add the following into it:

syn match pythonError "^\s*def\s\+\w\+(.*)\s*$" display
syn match pythonError "^\s*class\s\+\w\+(.*)\s*$" display
syn match pythonError "^\s*for\s.*[^:]$” display
syn match pythonError “^\s*except\s*$” display
syn match pythonError “^\s*finally\s*$” display
syn match pythonError “^\s*try\s*$” display
syn match pythonError “^\s*else\s*$” display
syn match pythonError “^\s*else\s*[^:].*” display
syn match pythonError “^\s*if\s.*[^\:]$” display
syn match pythonError “^\s*except\s.*[^\:]$” display
syn match pythonError “[;]$” display
syn keyword pythonError         do

Now that you have the basics covered, lets get more complicated checking added. Add these 2 lines to your .vimrc so you can type :make and get a list of syntax errors:

autocmd BufRead *.py set makeprg=python\ -c\ \"import\ py_compile,sys;\ sys.stderr=sys.stdout;\ py_compile.compile(r'%')\"
autocmd BufRead *.py set efm=%C\ %.%#,%A\ \ File\ \"%f\"\\,\ line\ %l%.%#,%Z%[%^\ ]%\\@=%m

You will have the ability to to type :cn and :cp to move around the error list. You can also type :clist to see all the errors, and finally, sometimes you will want to check the syntax of small chunks of code, so we’ll add the ability to execute visually selected lines of code, add this snippet to your .vimrc:

python << EOL
import vim
def EvaluateCurrentRange():
eval(compile('\n'.join(vim.current.range),'','exec'),globals())
EOL
map <C-h> :py EvaluateCurrentRange()

Now you will be able to visually select a method/class and execute it by hitting “Ctrl+h”.

Browsing the source

Moving around the source code is an important feature in most IDE’s with their project explorers, so to get that type of functionality in vim we grab the Tag Listplugin. This will give you the ability to view all opened buffers easily and jump to certain method calls in those buffers.

The other must-have feature of an IDE when browsing code is being able to open up multiple files in tabs. To do this you type :tabnew to open up a file in a new tab and than :tabn and :tabp to move around the tabs. Add these to lines to your .vimrc to be able to move between the tabs with ALT+LeftArrow and ALT+RightArrow:


map <silent><A-Right> :tabnext<CR>
map <silent><A-Left> :tabprevious<CR>

Debugging

To add debugging support into vim, we use the pdb module. Add this to your ~/.vim/ftplugin/python.vim to have the ability to quickly add break points and clear them out when you are done debugging:

python << EOF
def SetBreakpoint():
    import re
    nLine = int( vim.eval( 'line(".")'))

    strLine = vim.current.line
    strWhite = re.search( '^(\s*)', strLine).group(1)

    vim.current.buffer.append(
       "%(space)spdb.set_trace() %(mark)s Breakpoint %(mark)s" %
         {'space':strWhite, 'mark': '#' * 30}, nLine - 1)

    for strLine in vim.current.buffer:
        if strLine == "import pdb":
            break
    else:
        vim.current.buffer.append( 'import pdb', 0)
        vim.command( 'normal j1')

vim.command( 'map <f7> :py SetBreakpoint()<cr>')

def RemoveBreakpoints():
    import re

    nCurrentLine = int( vim.eval( 'line(".")'))

    nLines = []
    nLine = 1
    for strLine in vim.current.buffer:
        if strLine == ‘import pdb’ or strLine.lstrip()[:15] == ‘pdb.set_trace()’:
            nLines.append( nLine)
        nLine += 1

    nLines.reverse()

    for nLine in nLines:
        vim.command( ‘normal %dG’ % nLine)
        vim.command( ‘normal dd’)
        if nLine < nCurrentLine:
            nCurrentLine -= 1

    vim.command( ‘normal %dG’ % nCurrentLine)

vim.command( ‘map <s-f7> :py RemoveBreakpoints()<cr>’)
EOF

With that code you can now hit F7 and Shift-F7 to add/remove breakpoints. Then you just launch your application with !python % (percent being the current file, you can declare your main file here if its different).

Another tweak I use is to have my vim inside screen with a horizontal split, that way I can see the python interpreter and debug while still having vim there so I can easily fix my code.

Snippets

A great time saver with standard IDE’s is code snippets, so you can type a few key strokes and get a lot of code out of it. An example of this would be a django model, instead of typing out the complete declaration you could type ‘mmo<tab><tab>’ and have a skeleton of your model done for you. To do this in vim we grab the Snippets EMU plugin.

Check out a great screencast of snippetsEmu in action here

Emacs

Here is a great post on how to do the same with Emacs.