Test environment.

  • Dell R210II platform with Intel i3-2100 CPU (HT off) and 16GB DDR3-1333 memory running FreeBSD 9.1-STABLE
  • WDC WD1003FBYX hard drive as a system drive
  • Corsair Neutron SSD as a file system for a MySQL database
  • ExaBGP process (the latest source code from ExaBGP repository) run by pypy 2.0.2
  • collector.py run by python 2.7.5
  • MySQL 5.5.32 database (with skip-concurrent-insert and skip-innodb configuration options)
  • Juniper and Cisco routers as BGP peers

The processing time was measured as a time difference between the first ExaBGP log entry “Connected to peer neighbor” and to the time when processed routes counters reached 462k prefixes for each full IPv4 peer.

Test 1 – ExaBGP without collector.py – 1 full IPv4 peer.

  • 1st run – 148s / 2m28s
  • 2nd run – 168s / 2m48s
  • 3rd run – 146s / 2m26s

Test 2 – ExaBGP without collector.py – 2 full IPv4 peers.

  • 1st run – 79s / 1m19s
  • 2nd run – 94s / 1m34s
  • 3rd run – 91s / 1m31s

Test 3 – ExaBGP without collector.py – 6 full IPv4 and 6 full IPv6 peers.

  • 1st run – 184s / 3m4s
  • 2nd run – 190s / 3m10s

Test 4 – ExaBGP with collector.py, but without SQL operations – 1 full IPv4 peer.

  • 1st run – 69s
  • 2nd run – 46s
  • 3rd run – 48s

Test 5 – ExaBGP with collector.py, but without SQL operations – 2 full IPv4 peers.

  • 1st run – 62s
  • 2nd run – 64s
  • 3rd run – 61s

Test 6 – ExaBGP with collector.py, but without SQL operations – 6 full IPv4 and 6 full IPv6 peers.

  • 1st run – 195s / 3m15s
  • 2nd run – 198s / 3m18s

As above, but with exabgp.cache.attributes=false and exabgp.cache.nexthops=false.

  • 1st run – 204s / 3m24s
  • 2nd run – 213s / 3m33s

Memory usage after processing all prefixes with attributes and nexthops caching enabled.

 7814 nobody        1  20    0   223M   163M select  1   1:58  0.00% pypy2.0   < - ExaBGP
 8624 nobody        1  20    0 41240K  7948K piperd  0   2:24  0.00% python2.7 <- collector.py

Memory usage with attributes and nexthops caching turned off.

 7761 nobody        1  20    0   190M   130M select  1   2:04  0.05% pypy2.0   < - ExaBGP
 8234 nobody        1  20    0 41240K  7948K piperd  0   2:24  0.00% python2.7 <- collector.py

Test 7 - ExaBGP with collector.py + SQL operations - 1 full IPv4 peer.

  • 1st run - 374s / 6m14s
  • 2nd run - 368s / 6m8s
  • 3rd run - 368s / 6m8s

Test 8 - ExaBGP with collector.py + SQL operations - 2 full IPv4 peers.

  • 1st run - 760s / 12m40s
  • 2nd run - 761s / 12m41s
  • 3rd run - 755s / 12m35s

Test 9 - ExaBGP with collector.py + SQL operations - 6 full IPv4 and 6 full IPv6 peers.
Number of prefixes processed by each full IPv4 peer:

  • after 10 minutes - between 93k and 108k prefixes
  • after 20 minutes - between 211k and 215k prefixes
  • after 30 minutes - between 325k and 333k prefixes
  • after 40 minutes - between 448k and 452k prefixes

Total processing time - 42m15s

For comparison an Atom N2800 based platform with 2GB of memory and 2 SATA drives in RAID1 needed more than 6 hours to process prefixes from 5 full IPv4 peers.

Some advice how to increase the overall performance of ExaBGP and collector.py:

  • split full BGP peers to separate ExaBGP processes, especially on multi-core systems
  • disable unnecessary logging in ExaBGP, collector.py and MySQL database
  • MySQL database and file system tuning

Leave a Reply

Your email address will not be published. Required fields are marked *