I am trying to build a template for Perl scripts so that they would do at least most of the basic things right with UTF-8 and would work equally well on Linux and Windows machines.

One thing in particular escaped me for a while: the difficulty of passing UTF-8 strings as arguments to system commands. It seems to me that there is no way not to have arguments double UTF-8 encoded before they reach the shell (that is, I understand that there is a layer that ignores that the command and its arguments are already properly UTF-8 encoded, takes it for Latin-1 or something of the sorts, and encodes it again as UTF-8). I could not find a way to cleanly avoid this layer of encoding.

Take this script:

#!/usr/bin/perl

use v5.14;

use utf8;
use feature 'unicode_strings';
use feature 'fc';
use open ':std', ':encoding(UTF-8)';
use strict;
use warnings;
use warnings FATAL => 'utf8';

use constant IS_WINDOWS => $^O eq 'MSWin32';

# Set proper locale
$ENV{'LC_ALL'} = 'C.UTF-8';

# Set UTF-8 code page on Windows
if (IS_WINDOWS) {
  system("chcp 65001 > nul 2>&1");
};

# Use Win32::Unicode::Process on Windows
if (IS_WINDOWS) {
  eval {
    require Win32::Unicode::Process;
    Win32::Unicode::Process->import;
  };
  if ($@) {
    die "Could not load Win32::Unicode::Process: $@";
  };
};


# Show the empty directory
print "---\n" . `ls -1 system*` . "---\n";

my $utf = "test-тест-מבחן-परीक्षण-