Volunteers with lots of different perl versions and OSes run software that monitors CPAN for new uploads, and then automatically downloads them and runs their automated tests, and uploads the results.

Here's more on CPANTesters: http://wiki.cpantesters.org/wiki/WhatIsCPANTesters

And here's an example of the results for a popular module: http://matrix.cpantesters.org/?dist=Moose+2.1600

Quite short version:

Almost every perl module has a set of test scripts that can usually be called up by running make test. This happens by default when installing something with CPAN.pm. The output of this can be captured by CPAN.pm and sent on to a receiving server. Said server can be anything, but in almost all cases is the cpantesters metabase server, which integrates the result into a database which contains and summarizes the results of tests for libraries, by library version, operating system and perl version. In addition to people configuring their CPAN.pm that way and sending reports for everything installed, there are also many people who have set up servers that work on testing the newest version of every library uploaded to CPAN.