Descriptive Stats with C++ and Boost.Accumulators

Boost is a collection of mostly header-only libraries which make mere mortals like myself free-ride on years of accumulated wisdom of talented C++ programmers. It is also rather straightforward to use it on Windows which is a benefit for people like me who does stuff on all three major operating systems.

One Boost library I hadn't used at all was Boost.Accumulators. The other day, I was wondering how much work it would take to write a small program to calculate simple descriptive statistics on a column of numerical data. … more

Another Lenovo 3000 N100 bites the dust

It happened again.

My Lenovo 3000 N100 died.

This time, it happened while I was working on something. I was typing something, nothing particularly demanding was going on in the background, and, all of a sudden the screen went blank and all the indicator lights turned off. About a dozen different attempts at bringing it back to life did not pan out. … more

How to sum data from multiple files in Perl?

Stackoverflow is not that much fun any more, but I still look there to see if there are any interesting questions. Sometimes, relatively ordinary looking questions lead to useful insight. One such example is the question titled Perl : Adding 2 files line by line. … more

Manipulate %PATH% in the current Windows cmd.exe window with help from Perl

I often find myself working in one ConEmu window for extended periods of time. Sometimes, I end up adding a bunch of directories to the %PATH% during a session. And, more often than not, I end up wanting to just remove one or two directories from the %PATH% and continue working. For example, I might want to switch to using Cygwin's git instead of the MinGW git installed alongside Visual Studio 2015.

So, I wrote a quick Perl script to do that. It outputs a string so I can grab that and set the path in the current shell. … more

An old Visual C printf bug affects Perl

Perl doesn't yet build with Visual Studio 2015 tools out of the box on Windows. I have been using VS 2013 to build my perls on Windows 10. A few days ago, I noticed that a C program and an equivalent Perl program were printing floating point numbers differently ... more

Define constants for use by other modules in Perl

We regularly encounter lists of constants. Quick examples are HTTP status codes, Virtual Key Codes, lists of error codes and many many more. Regardless of the language you are using, you need a way to use those constants via friendlier symbolic names rather than literal numbers ... more

Display git branch in Windows command prompt

David's post "Displaying the Git branch in the terminal prompt with and without Perl" reminded me of a batch file I use for the same purpose. Of course, vanilla cmd.exe does not allow dynamic updating of the prompt, so it is rather more limited, but it can still be useful if you spend most of your session within one repo ... more

Perl is not the fastest language

Back in December 2015, I noted that perl was getting faster. Perl 5.24 was released a few days ago. The upgrade enabled perl to run the nbody benchmark 33% faster ... more

Looking back at the scary DJIA crystal ball

Back in 2014, a chart, purporting to show great similarities in the movements of the DJIA at that time with the period preceding the crash of 1929, was making the rounds on Wall Street ... more

MacGyvering HTML email using Perl's Template Toolkit

Producing HTML email is generally a frustrating endeavor akin to traversing a minefield one step at a time.

Today, I started thinking about how to do this sort of thing without writing any code, without manipulating any trees etc etc.

I know ... young hipsters these days probably have some gigabytes of Node infrastructure in their home directories with watchers, digesters, gulpers, notifiers and the like to do this kind of thing, but I have Perl and the Template Toolkit! ... more

A tar anomaly

A bug-fix to File::Which caused me to discover some unexpected (by-design) behavior of MinGW and Cygwin versions of the tar utility on Windows: Files may evaporate on extraction ... more

Perl is spreading to Windows

Installing git alongside Visual Studio 2015 Update 2 CTP brings perl 5.22.0 to Windows developers´ machines … more

Installing CommonMark Perl Module on Windows 10

I wanted to use cmark, the C reference implementation of CommonMark. Building the library and the command line utility using Visual Studio tools is straightforward enough. But, I ran into a few roadbumps on the Perl side of things … more

A first look at zsun WiFi SD card reader

zsun WiFi SD Card Reader, in a nutshell, is a WiFi access point with more flash (16MB) and RAM (64 MB) than most OpenWrt supported routers. Add the fact that you can plug in a rather sizeable micro SD card (I had a 16 GB one lying around, but I believe it takes up to 128 GB), and you have a matchbook sized computer with interesting possibilities … more

An explicit solution for d-digit excellent numbers in terms of factors of 10d - 1

I got a lot of great feedback after posting about excellent numbers on Hacker News. An anonymous user, going by the nickname Someone, noticed a way to re-write the equation for excellent numbers. … more

An excellent optimization story

I was able shave about 95% from the running time of a long running program searching through large sets of integers. While the particular problem might sound esoteric, this type of thing pops up in many scientific and financial contexts where multi-precision arithmetic libraries tend to be heavily utilized. … more

Rakudo Perl 6 was released on Christmas

So, I took it for a spin the morning after Christmas. Using Microsoft Visual Studio 2015 tools, it built without problems from a straight git clone (you might want to wait for the next Rakudo Star distribution release) … more

Perl6's newline translation behavior has been fixed

I am very happy to report that Bug #126881 for perl6: Newline handling is broken on Windows was recently resolved … more

Perl6's newline conversion behavior seems to be by design

It seems, from a distance, to a person not familiar at all with how Perl6 internals development works, that it was too hard to fix the issue properly by doing EOL conversion at input-output boundaries. Therefore, they seem to have settled for manipulating string literals in a program … more

Newline translation in Perl6 is broken

Perl6 seems to have done away with the convention of doing EOL translation at input-output boundaries. Instead, it seems to convert \n characters in strings to \r\n sequences which causes a large number of spurious test failures. I hope this behavior is not by design … (more)

Building Rakudo Star Perl6 distribution on Windows 10 with Microsoft Visual Studio tools

It's been more than a year since I last looked at Perl6.

Time flies. Pretty soon, it is going to be Christmas for the first time again. Well, OK, that didn't make much sense: What matters is the fact that there have been huge developments on the Perl6 front this year, and I would like to try to catch up… (more)

Free go faster stripes for your Perl programs

We all know benchmarks don't mean anything, but, let's admit, the Perl entry in the n-body benchmark is pretty slow. The fastest program (written in C++) solves the problem in less than 10 seconds whereas it takes Perl eighteen minutes, fully 115 times longer, to finish.

perl 5.23.5 improves on this … more

What happens to tests that are never run?

Version numbers are hard. Today, discovered that a test in DBD::SQLite had never run since a version check was added in April … (more)

Dive into the Global Historical Climatology Network data using Perl and SQL

The GHCN is the benchmark database which NOAA NCEI uses to make statements such as:

The combined average temperature over global land and ocean surfaces for October 2015 was the highest for October in the 136-year period of record, at 0.98°C (1.76°F) above the 20th century average of 14.0°C (57.1°F).

(more ... )

Download all the pieces of the full text of TPP, and combine them into a single PDF document

A few days ago, the full text was posted on the U.S. Trade Representative's web site, and on the web site of New Zeland Ministry of Foreign and Trade.

The first thing I noticed was the fact that both governments had decided the to post the so called "full text" of the treaty in bits and pieces: On the U.S. site, there are 238 individual pieces. New Zealand offers a ZIP archive that contains the 30 main documents (without annexes) in one handy archive, but that's still 30 files … (more)

A couple of long-lived bugs: Unchecked assumptions

I was really excited to be able to run cpan-outdated | cpanm on my Windows machine once again. This lead to the discovery of a couple of bugs in Dancer and Imager … more

Mobile internet access in Turkey for your computer and other devices

Reliable internet access as you travel through Turkey is a problematic proposition … It looks like a small, but very efficient operation in İstanbul is trying to change that. I had a chance to try out Alldaywifi, and, I can confirm that it works just as advertised… more

Recovering from a laptop disaster using a consciousness transplant

I tried to turn on the Lenovo, and I got nothing. Absolutely nothing.

I took out the battery, pressed the power button continuously for more than a minute … Nothing … Did that again, this time used the wall outlet … Nothing. There was absolutely no sign that the laptop was alive. After half an hour of trying various permutations of pressing the power button for an extended period of time, taking out the battery, putting it back in, connecting to mains etc etc, I was convinced my favorite computer was dead. … more

Slashing one's feet with tests, or, how to fix 2,950 test failures in one fell swoop

This morning's cpan-outdated | cpanm failed with a torrent of test failures from Pod::Simple.

The log file was so large that I was intimidated at first. Then, I switched to the build directory to see what was failing. Running nmake test filled my screen with a scrolling avalanche of test failures. When the dust settled, I noticed a pattern to the last few tests that were still visible on the screen: (more)

Playing with the TrueRNG 2 hardware random number generator

Recently, I came across TrueRNG 2 on Amazon. I made an impulse decision to take it for a spin. The design on TrueRNG 2 is based on the so-called avalanche effect. The device I received was in a black plastic enclosure with no markings on it except for a cheap sticker with TrueRNG2 / ubld.it printed over the background of an American flag. (more)

Test failures and directory separators: Take N

A routine update to Perl's Inline::CPP brings about another set of test failures due to the implicit assumption that the same character works as the directory separator on all platforms … (more)

Translating a Rosetta Code entry from Perl to C++

"If the latest C++ could improve on this Rosetta Code task entry for C++ then you might want to compare your improved C++ against the other scripting language solutions such as Perl, Python, Ruby, and Tcl." (more)

No more CPU upgrades for you!

It did not take too long to figure out I had screwed the heatsink on too tightly. It took even less time for me to strip three of the four Phillips screws holding the heatsink in place. The screws have springs around them, and, apparently, if you apply too much force, you're done!

What to do? (more)

Linode makes it hard to maintain uptime (in a good way :-)

Checking my Linode dashboard this morning, I noticed a little notification tucked away in the bottom right corner of the screen announcing the availability of a KVM upgrade … (more)

Can the creators of Quorum write programs in Quorum?

I finally broke down and downloaded Quorum to a Linux VM. I wanted to see what output their Quorum code produced, as I had failed to run it using their online widget.

Using the image of the reference Quorum code which they gave their participants in both studies, I typed in the following code, and put it in a text file. … (more)

Then they came for your Java and Ruby

Recently, brian d foy posted a challenge to rewrite a Perl example in Stefik et al. (2011). In that paper, the authors infamously proclaimed:

Results showed that while Quorum users were afforded significantly greater accuracy compared to those using Perl and Randomo, Perl users were unable to write programs more accurately than those using a language designed by chance.

Quorum seems to be their pet language which they are pushing to replace actual programming languages in widespread use in the industry. (more)

Upgrading my laptop's wireless card to an Intel 7260

Sometimes, I get the itch to tinker with hardware, and I have to scratch it — one hopes without breaking anything.

A few days ago, I discovered the easiest fix when I noticed extremely reasonably priced Intel Network 7260.HMWG.R cards on Amazon.

The itch was killing me. I ordered one, along with the bracket adapter. (more)

Multiple cohabitating versions of Perl on Windows

I just built my shiny new Perl 5.22.0 on my 64-bit Windows 8.1 system using Visual Studio 2013 Community Edition.

First, and foremost, thank you to all who contributes to the development of Perl. Also, thanks to Microsoft for making Visual Studio Community editions available.

While going through the motions of installing the various CPAN modules, the old issue of multiple cohabitating native Windows Perl distributions reared its head again (more)

C++: Walking the filesystem with Boost

Last time, I left after putting together a short but complete program that processed command line arguments, and showed a help message. (more)

C++: Parsing command line arguments with Boost

Still feeling excited about modern C++, I decided to install Boost with MS Visual Studio 2015 RC. Of course, most of Boost is header-only, but there are some libraries with binary components. One of those is Boost.Program_options whose purpose is to provide nice interface to reading program options from either the command line or a config file. (more)

Who is testing the tests?

I have no qualms with the basic premise of test-driven development. However, it doesn't take long before one realizes that any test that is even moderately complicated can itself become a source of bugs. At some point, it feels like the tests themselves are going to need test suites which themselves will need test suites etc ad infinitum. (more)

Golfing a word count program

Most interesting Perl programs are not one-liners. They free the programmer from the drudgery of building up and tearing down commonly used data structures to enhance the programmer's ability to focus on the real problem. It is refreshing to see that C++ now feels that way as well. (more)

Can C++ become your new scripting language?

Containers that free the programmer from the burdens of manual memory management, algorithms, lambdas, and even autovivification! Is C++ becoming straightforward enough to use in daily programming tasks? (more)

Sending faxes using Perl on Windows

A minimal script to send a fax using Perl on Windows. Of course, this assumes your computer actually has a fax modem installed. If not, the RNX-56USB looks like it would do the job. It works on both Linux and Windows.

Can programmers do Statistics?

I have a feeling that most programmer types assume Stats is trivial because the arithmetic involved in most of the basic concepts they have occasion to use is trivial.

In reality, people who assume Stats is trivial share the same lack of understanding pretty much everyone else has about basic Stats concepts. For example, many of them seem to assume that significant test statistic, by itself, means something about the truth of a hypothesis, or that a high correlation coefficient between two variables means one causes the other, or one can reliably predict future values one using the future values of the other.

Windows does have symlinks

Since Vista, one can use the mklink command to create symlinks. mklink is rather cumbersome to use because Windows has no sudo (sudowin aside), but symlinks do exist, and they are pretty convenient once you create them … (more)

Highlight Perl source code using PPI::HTML

A recent Stackoverflow question introduced me to the PPI::HTML module which uses the amazing PPI module to parse Perl source code, and associate CSS classes with the various elements.

What if you wanted to produce a self-contained chunk of syntax-highlighted Perl without depending on external CSS or JavaScript? In that case, you can resort to a somewhat grungy technique I use when I am generating HTML email: Post process the HTML to replace classes on elements with style attributes.

How you average numbers matters

Every now and then, I end up having to explain to skeptical people why it matters how their programs treat the numbers they ingest. With IEEE 754 and doubles, people seem to think that one can just will nilly add a bunch of numbers and average them, and get reliably accurate results … (more)

It all boils down to stat vs fstat from MSVCRT on Windows

Last time, I had just noticed that my perl's stat $filename and stat $fh did not agree on the modification time of a given file.

This turns out to be due to a difference in the way the underlying system calls on Windows behave. (more)

Time zone insanity on Windows

A bit of time zone insanity on Windows causes a spurious test failure with Mojolicious. (mode)

Fixing an oversight in Math::Int64

stdint.h is included if we are compiling with MinGW, but not if we are compiling with a Microsoft compiler. There is a good reason for that: Visual Studio up to version 2010 did not include stdint.h.

But, I am building this with VS 2013, so things get messed up. (more)

Do you uuencode? pack 'u' does!

Uuencoding, which I first used over 25 years ago, and forgot about soon after I stopped using Eudora on Windows 3, can be used to include encoded binary data in the __DATA__ section of a Perl file. (more)

Include image data in the data section of your Perl script or module

This example shows how to do it without adding new dependencies to your project … (more)

Fixing malformed XML with Perl's XML::Parser

I have more experience than I would like to recount being in the middle of a data processing pipeline. I often have to acquire data sets which have been produced using deity knows what kind of COBOL written several decades ago, then passed through several layers to produce what seems to be the cool data format of recent times. Owing to the fact that I am both an economist and a developer, I am often also the end user of said data sets, so I have a vested interest in getting those data sets to usable shape. I have had to resort to using JScript on locked down Windows Servers to process awful lookup tables put together as if the largest hard drives were still 44 Mb Miniscribes.

Frequenly, you find yourself having jumped through 1600 bureaucratic steps to actually get access to the data, and the upstream is either unresponsive to constructive criticism, or worse. In these circumstances, it falls on you to fix others' mistakes, and keep things moving, because it is not your job, say, to enforce the XML standard. (more)

Hands off that constant

In this Stackoveflow question titled "How do I programatically construct constant name and use value of constant?", we have someone who wants to be able to construct names of constants and look up their values.

A Perl programmer knows that if you want to look up values based on string keys, you should use a hash. (more)