Fixing the LS_COLORS problem and improving terminal colors
Posted: Mon May 16, 2022 9:21 pm
The LS_COLORS environment variable determines file colors during directory listings (ls colors, get it?). LS_COLORS is used to set a file's listing color based on a pattern match to the file name. The default file colors are set by the utility dircolors, and these colors are boring. Many people agree.
You can find some much nicer coloring schemes on the net. But despite this attention to detail, there isn't an easy way to change coloring schemes, or to even set them for that matter. Most people don't know about these other color schemes and just use the default.
To make this feature more accessible, I created the utility set_file_colors to graphically demo and change LS_COLORS among various fancy options. If you use this utility, you will be all set for beautiful and informative listings.
HOWEVER, stuffing these fancy coloring schemes into LS_COLORS frequently makes the environment variable too large. Environment variables weren't designed with this in mind. Having a large environment variable causes some programs to bomb with the message "ERROR: too long environment variables please use --rmenv", or words to that effect. Even worse, some Linux programs launched from a menuing system will silently fail to run. This is because stderr is redirected, and you can't see the error message.
I was trapped between having beautiful file listings and needing all programs to run without bombing.
I solved this problem by rewriting the GNU coreutils ls command and the tree program (the two main programs using LS_COLORS). My modified programs can now store a file name in LS_COLORS that contains the color scheme. Under this new system, which is a trivial modification of the old system, three new hidden files are added to each user's $HOME directory: .dircolors, .dircolors_selection, and .dircolors_setting. The .dircolors is a script that sets LS_COLORS based on the settings in .dircolors_setting. The file .dircolors_selection actually contains the LS_COLORS, as in echo $LS_COLORS > ~/.dircolors_setting. Thus, the color naming scheme remains unchanged. This makes the LS_COLORS variable very short, and programs no longer bomb because of the LS_COLORS being too long.
The magic is that the LS_COLORS variable may now express a path to a file. If LS_COLORS begins with a ';', which is malformed under the previous rules, the variable is interpreted as a file path. In this way, should the user choose a more traditional LS_COLORS setting, everything will work as before. Under the new system, LS_COLORS=';/home/user/.dircolors_setting' is a valid form.
You can find links and instructions for the updated programs in the build notes below.
While these new color schemes are nice, they don't cover a common situation. Sometimes, the files in a directory are of similar types, which are colored similarly. So you see a giant wall of similar colors. But what you really want is to see a big difference between file types. To address this issue, I added ls-rainbow and ls-normal programs. ls-rainbow compiles a histogram of the file types (by extension). It then gives those files pre-assigned vibrant colors that differentiate them. As a result, files which might appear the same color under a typical color scheme now have radically different colors. ls-rainbow resets LS_COLORS to accomplish this. ls-normal switches the LS_COLORS variable back to the previous setting. ls-normal is just an alias to
Code: Select all
export LS_COLORS=`dircolors` && [[ -f $HOME/.dircolors ]] && source $HOME/.dircolors; ls
The new programs also respect a new environment variable: TERM_BACKGROUND. You can set TERM_BACKGROUND to light or dark, and the color scheme will adjust automatically. This allows a single color setting to work when you change terminal background colors. UNFORTUNATELY, none of the terminal emulators know about this feature, so you will need to set it manually after changing your terminal background. In the future, I will be adding this feature to xfce4-terminal.
This new system is a temporary fix to solve the immediate problems. I can't stress that enough. I have a much more detailed scheme to address file coloring issues. However, this will need to wait until I can rewrite the coreutils in C++. The current coreutils is spaghetti code, and belongs to a different time. It needs a major re-think that doesn't break backward compatability. I only fixed the C code because I didn't realize how bad the current code was. In the amount of time I spent, I could have been well on the way to writing in C++. I see that there is a project to rewrite coreutils in the latest messiah-language (Rust). For various reasons, this isn't going to fly. I'm sure it will work, but it will be equally unmaintainable as the current situation, and no real improvement on the basic problems of the 21st Century that haven't been addressed by the GNUs.
Notes on building ls and tree:
The general idea is this: you will have the gnu coreutils and the tree package already installed. You download (or clone) those packages, substitute the new code, build, then clobber the existing ls and tree by copying ls and tree executables to the bin directory. You can either tell your package manager to lock the versions of these packages or you have to remember to rebuild and re-clobber as new versions of these packages come out.
The new ls can be built by overwriting ls.c with this new ls.c in any recent GNU coreutils and running make. So, it's a drop-in replacement. Similarly, Download tree, and overwrite color.c. Just build as normal. After chmod'ing the new ls and tree, copy the new ls to /bin/ls and /usr/bin/ls on a Linux system. Prerequisites for this new system are: bash and dircolors, both of which are typically installed. Copy the tree executable to wherever it normally lives. I am currently clobbering an installed package for coreutils and tree without difficulty.
While I was under the hood with ls, I thought I would add in a "directories only" mode to ls. ls -/ and ls -/R now list only directories, no files. I had previously been using ls -lF | grep / to accomplish this, but this is so much better. Directories without subdirectories don't get listed with ls -/R.
Notes on installing set_file_colors and its ancillary programs:
You need to modify your $HOME/.bashrc. Search for the line containing "dircolors -b ~/.dircolors". Change the .bashrc file to read as follows:
Code: Select all
test -r ~/.dircolors || eval "$(dircolors -b)"
test -r ~/.dircolors && source ~/.dircolors
You need to modify /etc/bash.bashrc by adding the following line to the end of the file
Code: Select all
trap "{ source "{HOME}/.dircolors" 2> /dev/null; }" SIGUSR1
Copy set_file_colors and ls-colors, ls-rainbow to /etc, and make them executable. Launching a yad program (set_file_colors) from the menu requires that the script be executable. Copy demo_colors.sh to /usr/local/bin and make it executable. Set_file_colors also requires an icon /usr/share/icons/gnome/48x48/categories/package_graphics.png, which is provided by the gnome icons package, which must be installed.
The main programs are called via shell aliases due to limitations of bash. You need to include the following lines (preferably) in your /etc/bash.bashrc (so everyone can use), or in your account ~/.bashrc or ~/.bash_aliases (assuming it is called by your bashrc somewhere):
Code: Select all
alias ls-rainbow='source /etc/ls-rainbow '
alias ls-normal='export LS_COLORS=`dircolors` && [[ -f $HOME/.dircolors ]] && source $HOME/.dircolors; ls'
alias set_file_colors='source /etc/set_file_colors';
If you want to access set_file_colors through a common desktop menuing system, you must copy set_file_colors.desktop to /usr/share/applications or your equivalent account directory so that the menuing system can find it. You may need to adjust a few things, but this should get you started.
Set_file_colors also calls tmux to demo the color schemes by opening windows with light and dark backgrounds. This was the only solution I could come up with to demonstrate the file colors before selecting. Hopefully, I can modify xfce4-terminal to serve this purpose and eliminate tmux in the future.
Because of the alias, ls-rainbow will be available to all users. It creates a histogram of files and tries to assign the greatest color differences among a limited list of colors that are perceptibly different. ls-rainbow has no help or man file, so I should mention that ls-rainbow also takes a second numerical argument, e.g. ls-rainbow N will construct the histogram on the top N directory levels. This is very useful for setting colors for the tree command.
ls-normal requires the new system, because it calls ~/.dircolors, which doesn't exist under the current system. Include in your .bash_aliases
Code: Select all
alias ls-normal='export LS_COLORS=`dircolors` && [[ -f $HOME/.dircolors ]] && source $HOME/.dircolors; ls'