The Perl Net::Netmask module is also worth checking out. It may not be better at aggregation but it does have other functions that could be helpful. I use the shortest match address lookup functions of Net::Netmask very heavily and have reproduced them in a R / C++ package. Jon On Thu, Oct 1, 2020 at 9:47 AM Tim Jackson <jackson.tim@gmail.com> wrote:

#!/usr/bin/perl use strict; use warnings; use Data::Dumper; use NetAddr::IP qw(Compact);

my @ips = ( '105.170.72.0/24', '105.170.73.0/24', '105.170.74.0/24' );

my @agged = aggregate(\@ips);

sub aggregate { my @naddr = map { NetAddr::IP->new($_) } @{$_[0]}; my @output = Compact(@naddr); return @output; }

On Thu, Oct 1, 2020 at 8:36 AM John Von Essen <john@essenz.com> wrote:

...

Sorry if this is slightly off-topic, but I am writing some code for a custom GeoDNS routemap. My starting data set is a raw list of /24 subnets, no prefix aggregation has been done. In other words, its the entire BGP routing table in /24 prefixes - tagged by Geo region. Each region is its own txt file with a dump of /24’s. As a result, these lists are HUGE. I want to aggregate the prefixes as much as possible to create a smaller routemap.

So right now it looks like:

... 105.170.72.0/24 brs 105.170.73.0/24 brs 105.170.74.0/24 brs 105.170.75.0/24 brs 105.170.76.0/24 brs 105.170.77.0/24 brs 105.170.78.0/24 brs 105.170.79.0/24 brs 105.170.80.0/24 brs 105.170.81.0/24 brs 105.170.82.0/24 brs 105.170.83.0/24 brs 105.170.84.0/24 brs …

and so on. Obviously, 105.170.72.0/24 thru 105.170.79.0/24 can be aggregated to 105.170.72.0/21 and so on. I normally use Perl, does anyone now if there is a perl module that will automatically do this prefix aggregation? I tried to write my code to do this, and its not trivial, just lookinh for a shortcurt. I did a breif glance at some CIDR related Perl cpan modules, and nothing has jumped out.

Thanks John

On Thu, Oct 01, 2020 at 02:15:01PM -0300, Marcos Manoni wrote:

Check https://github.com/job/aggregate6 (thank you, Job)

Marco Marzetti (PCCW) wrote an even faster compression tool! https://github.com/lamehost/aggregate-prefixes Both these python implementations are meant as replacements for ISC's vintage 'aggregate' Unix utility, with the notable difference that they also support IPv6. Example: job@bench ~$ pip3 install aggregate-prefix job@bench ~$ wc -l dfz_ipv4 810607 job@bench ~$ cat dfz_ipv4 | time aggregate-prefixes - | wc -l 141645 1m40.17s real 1m37.39s user 0m01.60s system Compressing the whole IPv4 DFZ prefix list takes only 100 seconds. Kind regards, Job

On Fri, Oct 02, 2020 at 03:39:00AM -0700, Randy Bush wrote:

...

Marco Marzetti (PCCW) wrote an even faster compression tool! https://github.com/lamehost/aggregate-prefixes

Both these python implementations are meant as replacements for ISC's vintage 'aggregate' Unix utility, with the notable difference that they also support IPv6.

ok, i gotta ask. has someone tested to see if they all produce the same result givem the same input? i do not mean to imply they do not. i just have to wonder.

Yes, of course. Marco and I collaborated on the tool's regression testing. job@bench $ aggregate6 < dfz_ipv4 | md5 066bfea49c4c20fed7d86d355044764a job@bench $ aggregate-prefixes < dfz_ipv4 | md5 066bfea49c4c20fed7d86d355044764a job@bench $ aggregate6 < dfz_ipv6 | md5 1193796d41cc47f32230da281e3ad419 job@bench $ aggregate-prefixes < dfz_ipv6 | md5 1193796d41cc47f32230da281e3ad419 Kind regards, Job

On 02.10.2020 at 12:03 Job Snijders wrote:

Marco Marzetti (PCCW) wrote an even faster compression tool! https://github.com/lamehost/aggregate-prefixes Both these python implementations are meant as replacements for ISC's vintage 'aggregate' Unix utility, with the notable difference that they also support IPv6.

Example:

job@bench ~$ pip3 install aggregate-prefix

job@bench ~$ wc -l dfz_ipv4 810607 job@bench ~$ cat dfz_ipv4 | time aggregate-prefixes - | wc -l 141645 1m40.17s real 1m37.39s user 0m01.60s system

Compressing the whole IPv4 DFZ prefix list takes only 100 seconds.

First time I uploaded/publish something to/on Github ... so please be kind: https://github.com/FvDxxx/pfxaggr In case you need it even faster (and can accept the little known issues and that it's old, ugly and never reviewed):

wc -l dfz-pfx-20201002-A-v4.txt 813542 dfz-pfx-20201002-A-v4.txt

time cat dfz-pfx-20201002-A-v4.txt | ./pfxagg -a1 > dfz-4-agg-pfx.log

real 0m1.034s user 0m0.909s sys 0m0.232s

time cat dfz-pfx-20201002-A-v4.txt | aggregate-prefixes > dfz-4-agg-pyth.log

real 1m11.691s user 1m10.879s sys 0m0.786s

diff dfz-4-agg-pyth.log dfz-4-agg-pfx.log wc -l dfz-4-agg-pyth.log dfz-4-agg-pfx.log 141754 dfz-4-agg-pyth.log 141754 dfz-4-agg-pfx.log 283508 total

Cheers, Markus