Foreign Characters

[Howard Rogers]

FreeNAS 9.3, and I am trying to store files on my FreeNAS datasets with foreign characters in their name (i.e., things like Ä,é,ë,ß,â,ö,ç). The files are shared via CIFS, and all my Windows and Linux clients can create such files and they display perfectly in their respective browsers. For example:

But, if I use the FreeNAS console to poke around in the shell, I get this:

Now, if I ssh into the freenas box, the same issue arises:

Code:

hjr@dirac ~ $ ssh root@192.168.137.215
root@192.168.137.215's password:
[root@freenas] ~# ls -l /mnt/MUSICVOL/SHARED_MUSIC/Classical/
total 14
drwxrwxrwx  5 nobody  guest  5 Aug 20 22:03 ./
drwxrwxr-x+ 3 root  guest  4 Aug 20 02:26 ../
drwxrwxrwx  2 nobody  guest  2 Aug 20 22:03 Arvo P??rt/
drwxrwxrwx  2 nobody  guest  3 Jun 26 22:29 F??r Lennart in memoriam (Kaljuste)/
drwxrwxrwx  2 nobody  guest  2 Aug 20 21:52 ??????????????/

But I can at least fix that:

Code:

[root@freenas] ~# /bin/bash
[root@freenas ~]# export LC_ALL=en_AU.UTF-8
[root@freenas ~]# ls -l /mnt/MUSICVOL/SHARED_MUSIC/Classical/
total 2
drwxrwxrwx  2 nobody  guest  2 20 Aug 22:03 Arvo Pärt
drwxrwxrwx  2 nobody  guest  3 26 Jun 22:29 Für Lennart in memoriam (Kaljuste)
drwxrwxrwx  2 nobody  guest  2 20 Aug 21:52 Äéëßâöç

However, when I try the same thing on the original console (rather than in a remote terminal connected via ssh), I don't achieve the same degree of success:

So my questions are:

1. Is anything wrong here? That is, does the file system genuinely store the correct characters and this is simply a matter of how the console can (or can't) display them accurately?
2. Is there anything I can do to get the console displaying the characters correctly? That is, how does one set LC_ALL in the .sh shell? (I only know to invoke the bash shell and issue export commands, I'm afraid).
3. Is the failure to display shown in that last screenshot just an artefact of the font that the console uses... and is there anything I can do about that?

I suppose the fundamental question is that #1: if my Windows and Linux clients are happy with what they store, and are happy displaying it back, is there any danger in allowing the backend to not display them correctly without mucking about with shells, locales and the like?

Any guidance appreciated.

 

[joeschmuck]

I'd submit a bug report for this. Be very descriptive in what FreeNAS software you are running and language/character set you have setup.

 

[Ericloewe]

I don't think there's a way around this with most shell's default settings.

It's purely a cosmetic issue (a usability issue too, since identification and input are made harder), so no need to worry. ZFS is correctly storing filenames (just don't try the stupid Unicode emojis).

 

[solarisguy]

setenv LC_ALL en_US.UTF-8

or if you had changed your shell

export LC_ALL=en_US.UTF-8

 

[Howard Rogers]

I appreciate all the replies.

@solarisguy: I did the setenv command you mentioned and managed to change Arvo P??rt into Arvo Pät, which doesn't seem that great an improvement :) ...I suspect the console font isn't up to displaying foreign characters very well. Wonder if there's a way to change it: I always prefer Solaris' use of a serif monospace font anyway, and generally change my Linux terminal fonts to Century Schoolbook Mono BT anyway.

@Ericlowe: I was hoping someone would say that ZFS was fine and it's purely cosmetic. It certainly seems that way from my clients' perspective, anyway.

Thanks guys. If you know how to change the console font, would appreciate it...

 

[solarisguy]

I am sorry, the US is clearly not superior to Australia, as far as the UTF-8 display is concerned

I had missed that you are using 9.3. When using 9.2.1, the console is using some software font that is UTF-8 aware and has lots of European characters.

 

[Howard Rogers]

I maybe have half an answer. In /etc/ttys, the main console terminal is set to 'unknown', with virtual terminals set to cons25 -which uses code page 437, which is the original IBM ASCII character set, and thus has line drawing characters (such as ├) where I need normal letters with diacriticals. So I changed that to read "cons25l1", which uses an ISO-8859-1 character set, which should have the necessary umlauts and accents... but still no joy. Not on the console itself, anyway (the shell you get in the web browser has always been displaying things correctly, even before I tried this /etc/ttys change). It is possible that I additionally need to add a font directive in the /etc/rc.conf file, but I'm a bit more wary of mucking about there.

Update: I added the fonts lines into /etc/rc.conf, anyway -and it still made no difference. Then I noticed that .profile in the root's home directory was setting a terminal type to cons25, and made that cons25l1... and still no difference.

Pesky things, these font/character set issues!