At the high level I usually work at, writing a Python script that runs on both Windows and Unix-like OSes. I ran into a couple surprises, however, when trying to generate a number of shell scripts from Windows designed to run with WSL.
We can setup the basic outline of a script that will write a shell script. Our goal is to run a command which requires the specification of an input and output file. Thus, the shell script will look something like (though with the same program running with a variety of inputs/outputs):
# desired output file program -N -p /mnt/c/path/to/source1.txt /mnt/c/path/to/source1.out
(In reality, the actual case is a bit more complex, involving varying the parameters and outputs…but this is enough to address the issues.)
with open(path, 'w') as out:
for file in iterpaths(): # iterpaths is a function that returns unix-styled `pathlib.Path` objects
out.write(f'program -N -p {file} {file.with_suffix(".out")}\n')
In the above code, let’s assume that iterpaths() returns pathlib.Path objects, though WSL-styled (like /mnt/c/.../source.txt). Note also the use of the pathlib.Path.with_suffix function which will replace the suffix (i.e., the extension) with .out, so the resulting file will be source1.out (replacing the .txt with .out.
The output from the above code, however, looks wrong to begin with, including forward slashes in the path rather than backslashes:
program -N -p \mnt\c\path\to\source1.txt \mnt\c\path\to\source1.out
The reason for this is that pathlib has output these paths using the operating system’s separator (which, in this case, is Windows). We can force it to conform to Unix paths using pathlib.Path.as_posix():
with open(path, 'w') as out:
for file in iterpaths():
out.write(f'program -N -p {file.as_posix()} {file.with_suffix(".out").as_posix()}\n')
After running the code, we can open up the resulting shell script, and everything looks right — we now have forward slashes:
program -N -p /mnt/c/path/to/source1.txt /mnt/c/path/to/source1.out
Unfortunately, if we peak at the line endings, we will see both a carriage return (CR) and newline (LF)(‘\r\n’). Running the code will insert a 0x0d after each output filename. We can re-write how the shell script actually looks:
program -N -p /mnt/c/path/to/source1.txt /mnt/c/path/to/source1.out[CR][LF]
Here, WSL will interpret the [LF] as a newline (as it should), but will interpret [CR] as the last character of the output path, creating a file called /mnt/c/path/to/source1.out[CR] rather than /mnt/c/path/to/source1.out.
To solve this problem, we need to direct Python to not use the Windows-specific interpretation of \n, since it will place \r\n instead. We can resolve this problem by adding a newline='' parameter to the open(path, 'w') function. This directs Python to not use any OS-specific interpretations, but write exactly what is included in the string. Thus, \n would be interpreted as \n ([LF]) instead of \r\n ([CR][LF]). (We can see a similar requirement when opening CSV files for writing: Python requires the file to be opened with this newline='' parameter since a CSV spec defines a CSV file as not containing any carriage returns, just newlines.)
Here’s our final bit of code:
with open(path, 'w', newline='') as out:
for file in iterpaths():
out.write(f'program -N -p {file.as_posix()} {file.with_suffix(".out").as_posix()}\n')
This gives us the expected output. Now, we can run this script on WSL without issue.