I turned them off completely. I found that the line numbers and file position in the status line was more than enough to get my bearings, and it looks nicer without the scrollbars.
In ~/.gvimrc:
" no toolbar
set go-=T
" no right scrollbar
set go-=r
" no left scollbar
set go-=L