• Skip to primary navigation
  • Skip to main content
  • Skip to primary sidebar

Chrispian H. Burks

coder. maker. writer. photographer.

  • Home
  • Dreams
  • Dev Links
  • About Me
  • Contact

Quick vi tip: Show Hidden Characters

Published on: July 2, 2020

I do a lot of development in linux/unix. In fact, that’s where all my development is, but I spend a fair amount of time in the “CLI” or command line interface. I use putty and ssh in to edit files and look at logs and just generally do what I do.

Today I ran into a problem with a file and I couldn’t figure out what was wrong. I kept getting an error and the spacing was off so I started suspecting a hidden character. vi, the command line editor I use in linux can show you hidden characters. Just type:

:set list

And bingo, hidden characters are revealed.

Turning them back off is just as easy:

:set nolist

Write that down, it comes in handy.

Updated 2/25/10: Thanks to Stephen in the comments, I’ve updated the above commands. I had the : in the wrong place!

Filed Under: Dev

About Chrispian

My name is Chrispian and I like to make things. I'm a web developer and have been working in the computer / internet field for almost 26 years now. I also draw, write, make things, take pictures and love figuring stuff out. Read More

Reader Interactions

Comments

  1. Greg Bergeson says

    July 30, 2008 at 1:50 pm

    Thanks for this info on vi. I didn’t want to drag out the books and figure out which option it was again! I really needed this for file conversion between Mac OS and Windows OS (the infamous carriage return versus line feed issue). Thanks again!

  2. Stephen says

    February 16, 2010 at 11:08 am

    You don’t need to post my response. I think you meant “:set list” and “:set nolist” instead of “set: list”. I got an error when I typed “set: list” and “:set: list”. Thought yuo might want to know so you can fix your post and save others the headache of figuring it out.

    This post was very helpful never the less. Thanks!

  3. Chrispian says

    February 25, 2010 at 3:50 pm

    Stephen, so right! Totally typo’d that. Updating it now.

  4. Eric says

    March 10, 2010 at 8:39 am

    Once you find the characters you want to remove you can use:
    :%s///g to remove everything globally. The way to type those pesky special characters in vi (ie ^M) is to use ctrl-v then ctrl-m You then substitute the m for whatever letter you need.

    Hope that helps!

  5. Eric says

    March 10, 2010 at 8:42 am

    Sorry the posting left out some of my characters. A sample of the above technique would be :%s/^M//g

    Sorry for the mixup!

  6. rpp says

    October 19, 2010 at 12:53 pm

    :%s/.$//g

    will take care of you dos file carry over ^M
    cheers\r

  7. Boris says

    December 23, 2010 at 8:19 am

    Is there a way to show the hidden \t characters?

    Thanks,

  8. Mikkel says

    January 31, 2011 at 5:24 am

    Also, if you like me would like a key-binding for this, you can use the :set list! version. Notice the “!” at the end. This will toggle list on and off.

  9. Santhi says

    June 7, 2011 at 12:30 pm

    This is a very useful command. Thanks for sharing this.
    We were having troubles running a service using a config file as the config file has hidden characters. I was able to find out the hidden characters in the file using this command and it resolved out issue.
    Thanks very much!

  10. Josh Lee says

    August 2, 2011 at 7:46 am

    I found out today that :set list in vim doesn’t actually always show dos characters, only sometimes. To make 100% sure you need to run “cat” on the file with the two flags “v” and “T”. So if your file is named foo.bar, you’d type in: “cat -vT foo.bar”

  11. brian says

    October 6, 2011 at 2:53 pm

    so much easier then getting up and going to my bookshelf to break out the o’reillys book. thanks

  12. Scott says

    October 7, 2011 at 12:48 pm

    Thank you for the Vi tip. I have been pulling my hair out over some CSS that is not rendering in the browser. Here is the CSS:

    .quiz-button-go:hover{
    color:#ddffff;
    background:#28e07e;
    }

    Simple right?! The problem is the color is not registering, but the background is. But, when I delete the spaces between the newline and “color”. it works perfectly.

    The CSS in question is CSS that I have cut and pasted from my browser into Vim.

    When I use :set list, what I get is

    44.quiz-button-go:hover{$
    45    $
    46 ^Icolor:#ddffff;$
    47 ^Ibackground:#28e07e;$
    48 }$

    I’m assuming ^I is tab and $ must be carriage return? Looks fine, right?

    Do you know of any other way to view any special hidden chars?

    I repeat, if I delete those few spaces, it works perfectly. So I know it’s not a CSS programming thing. This is totally bizarre.

    I’ve run into this kind of problem maybe twice in the last 4 months. It’s a real PITA because the text is indeed correct and hunting down this kind of bug is tough…

  13. Bradley Rentfrow says

    November 30, 2011 at 4:40 pm

    This is what I found to removed the pesky ^M’s that mess up your script:
    :e ++ff=dos
    :setlocal ff=unix
    :w

    When you run cat -vT or run the script the ^M’s should be gone.
    Basically it saves the file back into the UNIX format and scripts the DOS carriage returns from the file.

  14. WeatherARC says

    December 14, 2011 at 3:06 pm

    Thanks for the post. I now just have to decipher what I am seeing. Hmmm running :set list and then seeing some ^I or $ means what?

  15. Deniz Gezmis says

    June 24, 2012 at 10:15 pm

    If you happen to run Slackware Linux the following command should take care of those DOS related characters smoothly.

    fromdos output_file.txt

  16. Michael Monteith says

    August 9, 2012 at 10:03 am

    On some systems the vi -b option works great. Here is a good note on the subject
    http://www.devdaily.com/blog/post/linux-unix/vi-editor-binary-mode-remove-control-m-characters

  17. JoseMendoza says

    October 5, 2012 at 4:19 pm

    Thanks for the post, it definitely help to save the reference, since I only use it like once in a while. :-)

  18. Terry Mackie says

    October 26, 2012 at 1:21 pm

    I was using diff and by eye balling the code, I couldn’t figure out why 2 lines were different. I used set list and saw the 2 lines each had a space at the end. I removed the spaces and diff was sucessful.

  19. Hopping Bunny says

    July 5, 2013 at 4:01 am

    Helped me fix a strange problem that cropped up after I added some comments to a shell script !

    Go figure.

  20. TD47 says

    July 8, 2013 at 8:37 am

    Of course, if it is just the pesky ^M from a windows file that is the issue, there is always the good old “dos2unix” converston tool:
    dos2unix -o a.xtx b.txt

    That will automatically strip out the DOS line feed (carriage return code) stuff. HTH.

  21. Brian S says

    May 11, 2019 at 10:02 am

    Another command to find odd and pesky unknown characters is to type “od -cb foobar” where foobar is your file name this will show the file with the character you see and the octal code for that character. The Octal Dump (od) program has many features and options, “man od” for more.

  22. Chrispian says

    May 12, 2019 at 12:57 pm

    Brian, awesome tip! Thanks for sharing it. I need to compile some of the stuff people have added here to the post.

  23. Paul says

    May 17, 2019 at 2:26 pm

    for a quick look at special chars, try this

    :1,$l

    the last character is an “ell” not a “one”

    :1,$p prints the standard chars but not special

  24. Mi Sa says

    June 20, 2019 at 2:49 pm

    This is a great cheat sheet
    http://www.lagmonster.org/docs/vi2.html

  25. Chrispian says

    June 20, 2019 at 9:54 pm

    Mi, that’s awesome! Thanks for sharing.

  26. henry lai says

    December 2, 2019 at 3:41 pm

    dos2unix -n windows.txt unix.txt

  27. Sal Ferrarello says

    March 19, 2020 at 5:56 am

    I was just telling someone about this feature and I’m glad I found this post to share with them – thank you!

    Additionally, you can define which hidden characters are displayed and how they appear.

    I use Tim Pope’s Vim plugin vim-sensible, which sets the following.

    set listchars=tab:>\ ,trail:-,extends:>,precedes:<,nbsp:+

    Thanks again for this post!

  28. Paola Simonian says

    November 26, 2020 at 6:05 pm

    Hi! Thanks a lot for all the info!

    I’m still trying to remove the $ that I’ve found… but I’ve tried :%s/$//g and didn’t work. What am I doing wrong?

    Thanks!
    P

  29. Steve says

    June 1, 2022 at 5:21 am

    Wow! 30 years of Unix/Linux and never knew that. My life just got slightly better.

  30. Buddy says

    July 24, 2022 at 7:45 pm

    Just what I needed a week ago before I rekeyed 4 script files. Now I KNOW I’m not crazy. No worries and and thanks to you that won’t happen again.

Leave a Reply

Your email address will not be published. Required fields are marked *

Primary Sidebar

My name is Chrispian and I like to make things. I'm a web developer and have been working in the computer / internet field for almost 26 years now. I also draw, write, make things, take pictures and love figuring stuff out. Read More

Let’s Connect

  • Facebook
  • GitHub
  • Instagram
  • LinkedIn
  • Twitter

Copyright © 2023 · Groovy 1976 Theme by Chrispian H. Burks · Built using Genesis Framework & WordPress Log in