Burp

From DcSharedWiki
Revision as of 23:49, 24 August 2014 by Pilou (talk | contribs) (typo)

Burp tests

Description

Version: 1.3.48 (debian package 1.3.48-4~bpo70+1)

Hardware configuration

Type CPU Disk RAM
Server Marvell Armada 370 (ReadyNAS 104) 2*WDC WD20EFRX-68EUZN0 (soft raid1 using mdadm) 512Mo
Client1 VIA Nano L2007 1600MHz 2*WDC WD20EFRX-68EUZN0 (soft raid1 using mdadm) 4Go
Client2 VIA Nano L2007 1600MHz 2*WDC WD20EFRX-68EUZN0 (soft raid1 using mdadm) 4Go

All hosts have gigabit NICs with properly cabled CAT 6 cables and all are connected to a Gigabit switch (Netgear GS108E).

Ethernet controllers and modules:

  • client: `Realtek Semiconductor Co., Ltd. RTL8111/8168B PCI Express Gigabit Ethernet controller (rev 06)` (controller), `r8168` (module)
  • server: `Marvell 88E1318 Gigabit Ethernet PHY` (controller), `pegasus` (module)

Performances

Network

  • `iperf` measures 890 Mbits/s between server and client.
  • Copying 244.31Go grom client to backup server using rsync took 36h 21m (`sent 189.41G bytes - received 25.48Mbytes - 1.45Mbytes/sec`).
  • Copying 212Go from client1 (VIA PadLock usable by OpenSSL) to backup server using `scp` (default cipher specification and MAC algorithm) took 8h 1m 48s (7.50Mo/s).

Encryption

  • SCP transfers of 698Mo from the client1 (VIA PadLock usable by OpenSSL) to the server using different cipher and MAC algorithm:
|| cipher specification || MAC algorithm || Throughput || Duration ||
|| aes128-ctr || hmac-md5         ||  5.9MB/s || 01:59 ||
|| aes128-ctr || hmac-sha2-256    || 10.7MB/s || 01:05 ||
|| aes128-ctr || hmac-sha2-256-96 || 10.7MB/s || 01:05 ||
|| aes128-ctr || hmac-sha1        || 11.3MB/s || 01:02 ||
|| aes128-ctr || hmac-sha1-96     || 11.3MB/s || 01:02 ||
|| aes128-ctr || hmac-sha2-512    ||  8.3MB/s || 01:24 ||
|| aes192-ctr || hmac-sha2-256    ||  9.4MB/s || 01:14 ||
|| aes256-ctr || hmac-sha2-256    ||  8.4MB/s || 01:23 ||
|| aes128-cbc || hmac-sha1        || 14.0MB/s || 00:50 ||
|| aes128-cbc || hmac-sha2-256    || 11.8MB/s || 00:59 ||
|| aes192-cbc || hmac-sha2-256    || 11.5MB/s || 01:01 ||

Test using client1

Description Compression (`compression`) <<FootNote(Note: Extensions which are not compressed: bz, bz2, zip, gz, webm, mov, png, jpg, jpeg, rar, m4v, avi, xv, jar, tgz, mp4, mkv, ogg, flv)>> SSL compression (`ssl_compression`) Duration on client phase 1 phase 2 Data send Throughput (phase 2) Duration on server phase 3
full backup: 209.17Go gzip9 (default value) zlib5 (default value) 34h 17m 34s 07m 18s 34h 10m 18s 167.4Go 1.4Mo/s 06m 22s 02m 22s
full backup: 209.17Go None (`compression=gzip0`) zlib5 (default value) 14h 49m 29s 03m 57s 14h 45m 32s 209.17Go 4.0Mo/s 07m 04s 01m 09s
full backup: 214.27Go None (`compression=gzip0`) zlib5 (default value) 30h 05m 44s 7m 45s 29h 57m 59s 214.27Go 2.0Mo/s 06m 56s 01m 01s
full backup: 212.47Go None (`compression=gzip0`) None (`ssl_compression=zlib0`) 24h 19m 53s 04m 36s 24h 15m 17s 212.47Go 2.5Mo/s 06m 57s 54s
full backup: 213.99Go None (`compression=gzip0`) None (`ssl_compression=zlib0`) 30h 00m 45s 05m 47s 29h 54m 58s 213.99Go 2.0Mo/s 06m 59s 01m 05s
incremental backup: 1.5Go added None (`compression=gzip0`) None (`ssl_compression=zlib0`) 11m 30s 04m 36s 06m 54s 1.5Go 3.7Mo/s 16m 28s 01m 01s

Logs

Client 1 full backup 212.47Go 

2014-07-14 23:49:34: burp[5369] Client version: 1.3.48
2014-07-14 23:49:34: burp[5369] Begin phase1 (file system scan)
2014-07-14 23:54:10: burp[5369] End phase1 (file system scan)
2014-07-14 23:54:10: burp[5369] Begin phase2 (receive file data)
2014-07-16 01:09:27: burp[5369] End phase2 (receive file data)
2014-07-16 01:09:27: burp[5369] Begin phase3 (merge manifests)
2014-07-16 01:10:31: burp[5369] End phase3 (merge manifests)
2014-07-16 01:10:31: burp[5369] Backup ending - disconnect from client.
2014-07-16 01:10:31: burp[5369] Begin phase4 (shuffle files)
2014-07-16 01:10:31: burp[5369] Doing the atomic data jiggle...
--------------------------------------------------------------------------------
Start time: 2014-07-14 23:49:26
  End time: 2014-07-16 01:16:34
Time taken: 01:01:27:08
                         New   Changed Unchanged   Deleted     Total |  Scanned
                   ------------------------------------------------------------
            Files:    998898         0         0         0    998898 |   998898
      Directories:     74226         0         0         0     74226 |    74226
       Soft links:      4531         0         0         0      4531 |     4531
       Hard links:        16         0         0         0        16 |       16
      Grand total:   1077671         0         0         0   1077671 |  1077671
                   ------------------------------------------------------------

             Warnings:             0

      Bytes estimated:   228124139118 (212.46 GB)
      Bytes in backup:   228135345774 (212.47 GB)
       Bytes received:   228135345774 (212.47 GB)
           Bytes sent:             0
--------------------------------------------------------------------------------
2014-07-16 01:16:34: burp[5369] Backup completed.
2014-07-16 01:16:34: burp[5369] End phase4 (shuffle files)


Client 1 full backup 231.99Go 

2014-07-17 01:47:37: burp[4654] Client version: 1.3.48
2014-07-17 01:47:37: burp[4654] Begin phase1 (file system scan)
2014-07-17 01:53:24: burp[4654] End phase1 (file system scan)
2014-07-17 01:53:24: burp[4654] Begin phase2 (receive file data)
2014-07-18 07:48:22: burp[4654] End phase2 (receive file data)
2014-07-18 07:48:22: burp[4654] Begin phase3 (merge manifests)
2014-07-18 07:49:27: burp[4654] End phase3 (merge manifests)
2014-07-18 07:49:27: burp[4654] Backup ending - disconnect from client.
2014-07-18 07:49:27: burp[4654] Begin phase4 (shuffle files)
2014-07-18 07:49:27: burp[4654] Doing the atomic data jiggle...
--------------------------------------------------------------------------------
Start time: 2014-07-17 01:47:36
  End time: 2014-07-18 07:55:21
Time taken: 01:06:07:45
                         New   Changed Unchanged   Deleted     Total |  Scanned
                   ------------------------------------------------------------
            Files:    998898         0         0         0    998898 |   998898
      Directories:     74226         0         0         0     74226 |    74226
       Soft links:      4531         0         0         0      4531 |     4531
       Hard links:        16         0         0         0        16 |       16
      Grand total:   1077671         0         0         0   1077671 |  1077671
                   ------------------------------------------------------------

             Warnings:             0

      Bytes estimated:   228149960334 (212.48 GB)
      Bytes in backup:   229772435086 (213.99 GB)
       Bytes received:   229772435086 (213.99 GB)
           Bytes sent:             0
--------------------------------------------------------------------------------
2014-07-18 07:55:21: burp[4654] Backup completed.
2014-07-18 07:55:21: burp[4654] End phase4 (shuffle files)


Client1 full backup: 214.27Go 

2014-07-19 18:32:40: burp[9606] Client version: 1.3.48
2014-07-19 18:32:40: burp[9606] Begin phase1 (file system scan)
2014-07-19 18:40:25: burp[9606] End phase1 (file system scan)
2014-07-19 18:40:25: burp[9606] Begin phase2 (receive file data)
2014-07-21 00:38:24: burp[9606] End phase2 (receive file data)
2014-07-21 00:38:24: burp[9606] Begin phase3 (merge manifests)
2014-07-21 00:39:25: burp[9606] End phase3 (merge manifests)
2014-07-21 00:39:25: burp[9606] Backup ending - disconnect from client.
2014-07-21 00:39:25: burp[9606] Begin phase4 (shuffle files)
2014-07-21 00:39:25: burp[9606] Doing the atomic data jiggle...
--------------------------------------------------------------------------------
Start time: 2014-07-19 18:32:39
  End time: 2014-07-21 00:45:20
Time taken: 01:06:12:41
                         New   Changed Unchanged   Deleted     Total |  Scanned
                   ------------------------------------------------------------
            Files:    998898         0         0         0    998898 |   998898
      Directories:     74226         0         0         0     74226 |    74226
       Soft links:      4531         0         0         0      4531 |     4531
       Hard links:        16         0         0         0        16 |       16
      Grand total:   1077671         0         0         0   1077671 |  1077671
                   ------------------------------------------------------------

             Warnings:             0

      Bytes estimated:   230074235176 (214.27 GB)
      Bytes in backup:   230074235177 (214.27 GB)
       Bytes received:   230074235177 (214.27 GB)
           Bytes sent:             0
--------------------------------------------------------------------------------
2014-07-21 00:45:20: burp[9606] Backup completed.
2014-07-21 00:45:20: burp[9606] End phase4 (shuffle files)


Old tests

There was a network configuration problem: `r8169` module was used instead of `r8168` on the client.

Test using client1

Description Compression SSL compression Duration on client (phase 1/phase 2) Throughput (phase 2)
First backup: 210Go None (`compression=gzip0`) (default parameter `ssl_compression` not set) 37h 09m 30s (18m 56s/36h 50m 34s) 1.6 Mo/s
Second backup: 34.33Mo added None (`compression=gzip0`) (default parameter `ssl_compression` not set) 00h 18m 10s (16m 45s/1m 25s) 0.4 Mo/s
Third backup: 4.14Go (mp4 files) added None (`compression=gzip0`) (default parameter `ssl_compression` not set) 01h 19m 54s (11m 56s/1h 07m 58s) 1.03 Mo/s
Fourth backup: 4.24Go (mp4 files) added None (`compression=gzip0`) `ssl_compression=gzip0` 00h 58m 16s (16m 39s/41m 37s) 1.73 Mo/s

Test using client2

Description Compression SSL compression Duration on client (phase 1/phase 2) Throughput (phase 2)
First backup: None (`compression=gzip0`) (default parameter `ssl_compression` not set) 37h 06m 53s (15m 40s/36h 51m 13s)
Retrieved from "https://wiki.duckcorp.org//index.php?title=Burp&oldid=214"