STUTS Tech Blog


View and Remove Hidden Characters in Vim


Originally Published: 2020-04-03

Background

I have been adding my old articles to this site from an old Wordpress backup. Lazily, I've been copying & pasting the text I want into vim and adding/correcting HTML tags. This all seemed to be working fine until I viewed the render of some of my code blocks and saw the following  (dammit, it doesn't even render correctly here!) character appearing in so many places!

What are these and where did they come from?!

How To

It was initially surprising to me that these characters weren't in my source view (a less on the XML file) or in my vim session where I was pasting the text to.

To view the hidden characters, just run the following in vim (command mode)

:set list

Now some symbols should appear

I figured I had it sorted now, just highlight the symbol and insert it into a find/replace (s/+/ /g)... How wrong I was!

Now, it was tricky to identify what that character actually was and I enlisted the help of a colleague who managed to identify it with some emacs magic (don't ask, I'm on Team Vim) as a UTF-8 non-breaking space.

To replace this symbol specifically, run the following sed on the file

sed -i 's/\xC2\xA0/ /g' myfile

Note: Special thanks to this blog post which helped me with initially viewing the special characters.


If you have any feedback or issues regarding this then please do contact me