How can invalid assumptions remain uncovered through a battery of tests for such a long time?

In all my ranting about hard-coded file paths, and interpolated paths in regex patterns etc, one thing has been bugging me: All the modules involved look like they are passing all their tests with flying colors on CPANTesters.

One thing that is common to those platforms on CPANTesters is that none or almost none of them use Microsoft build tools. As far as I can tell, they are all various Strawberry Perl versions. Even if they used ActivePerl, they would probably be utilizing the MinGW toolchain.

Now, this in and of itself wouldn't cause Windows specific directory separators to disappear from view as File::Spec still correctly turns catfile qw(a b c) to a\b\c on those platforms.

But, let's say a module's Makefile.PL specifies:

  "test" => {
    "TESTS" => "t/*.t"

as many do. ExtUtils::MakeMaker::Unix::test treats this specially when the build tool is nmake:

    # have to do this because nmake is broken
    $tests =~ s!/!\\!g if $self->is_make_type('nmake');

When Strawberry Perl and ActivePerl use dmake, directory separators used in the values using the "TESTS" parameter remain untouched. Only when nmake is used are these converted to Windows style directory separators.

Given the gradual and almost complete disappearance of Microsoft build tools from the Perl ecology, this lulls module authors into a false sense of security by passing tests that assume all paths are Unixy paths.

Now that I have figured this out, I am happy.