20060126 Memory problem with VIM
Thursday January 26, 2006
Category: General
I love Vim, and I use it for all of my text editing until Eclipse came along. Now I program in Eclipse but still do a large portion of my text editing in Vim. Recently I was troubleshooting a problem with Thunderbird email, and had to open an Inbox file that was more than 200MB in size. With the past experiences with Vim opening large files, I did not hesitate before opening the Inbox file with Vim.

It took a long time (try more than 20 minutes) for Vim to load the whole file into memory, although I have 1GB of physical RAM on my notebook, but at least I was able to load the file and fix the problem. One thing that I observed is that the Mem Usage was 200MB, but the VM Size was around 400MB, double the Mem Usage.

Subsequently, when I tried to open another text file with Vim, the surprising fact is that my task manager still reports that Vim is trying to take up more than 400MB of memory to load the small 5KB text file!

A little research reveals that the _viminfo file stores the register for the last edit, and my last edit was the first line of the corrupted Inbox file, which I suspect to be a very long line. My _viminfo file was now also 400MB+ and every time I try to load Vim, it tries to load the registers into memory before allowing me to edit the text file.

Next logical thing for me to do was to either to delete the _viminfo file (which I think will be regenerated by Vim), or to open the _viminfo file to edit out the one long long line. With some time on my hands, I tried the second approach, just to see what happens.

It turned out that this was quite a bad idea. Now the file loads with more than 400MB under Mem Usage, and the VM Size climbed for more than 40 mins to around 1.3GB!! And the anti-climax was that at the end of 40mins, Vim loaded with the message that my machine has run out of memory. However, somehow Vim still loaded the _viminfo file, but I guess it has truncated a large portion of the file. I tried looking for the long line and it is not there, so I saved _viminfo, and noted that the size dropped to 2KB.

Next surprise is that when I closed Vim this time round, I was expecting everything to go back to normal. I was wrong because Vim still took a long time to close, and _viminfo went back to 400MB! Apparently after I saved _viminfo with the 2KB size, the application wrote back whatever it stored in memory and overwrote my changes.

Enough with the experimenting, I deleted _viminfo file and opened the small text file. Vim loaded with its usual fast speed, and a new (small) _viminfo file was generated when I closed Vim.

Lesson learnt: (1) If I opened a large file with a long line in Vim, I need to check the size of _viminfo before trying to open any other text files. (2) Never be so stupid as to try to open _viminfo with vim when _viminfo is very large. :)
Posted on (2006-01-25 22:19:20.0) Permalink Comments [0]
Trackback URL: http://jroller.com/trackback/dunpanic/Weblog/memory_problem_with_vim

Post a Comment:


Your Comment:

HTML Syntax: On

Please answer this simple math question

9 + 55 =