Burp: Difference between revisions

From DcSharedWiki
(typo)
(Fix units and spelling)
Line 17: Line 17:
| Marvell Armada 370 (ReadyNAS 104)
| Marvell Armada 370 (ReadyNAS 104)
| 2*WDC WD20EFRX-68EUZN0 (soft raid1 using mdadm)
| 2*WDC WD20EFRX-68EUZN0 (soft raid1 using mdadm)
| 512Mo
| 512MB
|-
|-
| Client1
| Client1
| VIA Nano L2007 1600MHz
| VIA Nano L2007 1600MHz
| 2*WDC WD20EFRX-68EUZN0 (soft raid1 using mdadm)
| 2*WDC WD20EFRX-68EUZN0 (soft raid1 using mdadm)
| 4Go
| 4GB
|-
|-
| Client2
| Client2
| VIA Nano L2007 1600MHz
| VIA Nano L2007 1600MHz
| 2*WDC WD20EFRX-68EUZN0 (soft raid1 using mdadm)
| 2*WDC WD20EFRX-68EUZN0 (soft raid1 using mdadm)
| 4Go
| 4GB
|}
|}


Line 41: Line 41:


* `iperf` measures 890 Mbits/s between server and client.
* `iperf` measures 890 Mbits/s between server and client.
* Copying 244.31Go grom client to backup server using rsync took 36h 21m (`sent 189.41G bytes - received 25.48Mbytes - 1.45Mbytes/sec`).
* Copying 244.31GB grom client to backup server using rsync took 36h 21m (`sent 189.41Gbytes - received 25.48Mbytes - 1.45Mbytes/sec`).
* Copying 212Go from client1 (VIA [[PadLock]] usable by OpenSSL) to backup server using `scp` (default cipher specification and MAC algorithm) took 8h 1m 48s (7.50Mo/s).
* Copying 212GB from client1 (VIA [[PadLock]] usable by OpenSSL) to backup server using `scp` (default cipher specification and MAC algorithm) took 8h 1m 48s (7.50MB/s).


==== Encryption ====
==== Encryption ====


* SCP transfers of 698Mo from the client1 (VIA [[PadLock]] usable by OpenSSL) to the server using different cipher and MAC algorithm (`aes128-ctr` and `hmac-md5` are the default settings on `Debian Wheezy`):
* SCP transfers of 698MB from the client1 (VIA [[PadLock]] usable by OpenSSL) to the server using different cipher and MAC algorithm (`aes128-ctr` and `hmac-md5` are the default settings on `Debian Wheezy`):
|| cipher specification || MAC algorithm || Throughput || Duration ||
|| cipher specification || MAC algorithm || Throughput || Duration ||
|| aes128-ctr || hmac-md5 || 5.9MB/s || 01:59 ||
|| aes128-ctr || hmac-md5 || 5.9MB/s || 01:59 ||
Line 69: Line 69:
| phase 1
| phase 1
| phase 2
| phase 2
| Data send
| Data sent
| Throughput (phase 2)
| Throughput (phase 2)
| Duration on server
| Duration on server
| phase 3
| phase 3
|-
|-
| full backup: 209.17Go
| full backup: 209.17GB
| gzip9 (default value)
| gzip9 (default value)
| zlib5 (default value)
| zlib5 (default value)
Line 80: Line 80:
|align="right" | 07m 18s
|align="right" | 07m 18s
|align="right" | 34h 10m 18s
|align="right" | 34h 10m 18s
|align="right" | 167.4Go
|align="right" | 167.4GB
|align="right" | 1.4Mo/s
|align="right" | 1.4MB/s
|align="right" | 06m 22s
|align="right" | 06m 22s
|align="right" | 02m 22s
|align="right" | 02m 22s
|-
|-
| full backup: 209.17Go
| full backup: 209.17GB
| None (`compression=gzip0`)
| None (`compression=gzip0`)
| zlib5 (default value)
| zlib5 (default value)
Line 91: Line 91:
|align="right" | 03m 57s
|align="right" | 03m 57s
|align="right" | 14h 45m 32s
|align="right" | 14h 45m 32s
|align="right" | 209.17Go
|align="right" | 209.17GB
|align="right" | 4.0Mo/s
|align="right" | 4.0MB/s
|align="right" | 07m 04s
|align="right" | 07m 04s
|align="right" | 01m 09s
|align="right" | 01m 09s
|-
|-
| full backup: 214.27Go
| full backup: 214.27GB
| None (`compression=gzip0`)
| None (`compression=gzip0`)
| zlib5 (default value)
| zlib5 (default value)
Line 102: Line 102:
|align="right" | 7m 45s
|align="right" | 7m 45s
|align="right" | 29h 57m 59s
|align="right" | 29h 57m 59s
|align="right" | 214.27Go
|align="right" | 214.27GB
|align="right" | 2.0Mo/s
|align="right" | 2.0MB/s
|align="right" | 06m 56s
|align="right" | 06m 56s
|align="right" | 01m 01s
|align="right" | 01m 01s
|-
|-
| full backup: 212.47Go
| full backup: 212.47GB
| None (`compression=gzip0`)
| None (`compression=gzip0`)
| None (`ssl_compression=zlib0`)
| None (`ssl_compression=zlib0`)
Line 113: Line 113:
|align="right" | 04m 36s
|align="right" | 04m 36s
|align="right" | 24h 15m 17s
|align="right" | 24h 15m 17s
|align="right" | 212.47Go
|align="right" | 212.47GB
|align="right" | 2.5Mo/s
|align="right" | 2.5MB/s
|align="right" | 06m 57s
|align="right" | 06m 57s
|align="right" | 54s
|align="right" | 54s
|-
|-
| full backup: 213.99Go
| full backup: 213.99GB
| None (`compression=gzip0`)
| None (`compression=gzip0`)
| None (`ssl_compression=zlib0`)
| None (`ssl_compression=zlib0`)
Line 124: Line 124:
|align="right" | 05m 47s
|align="right" | 05m 47s
|align="right" | 29h 54m 58s
|align="right" | 29h 54m 58s
|align="right" | 213.99Go
|align="right" | 213.99GB
|align="right" | 2.0Mo/s
|align="right" | 2.0MB/s
|align="right" | 06m 59s
|align="right" | 06m 59s
|align="right" | 01m 05s
|align="right" | 01m 05s
|-
|-
| incremental backup: 1.5Go added
| incremental backup: 1.5GB added
| None (`compression=gzip0`)
| None (`compression=gzip0`)
| None (`ssl_compression=zlib0`)
| None (`ssl_compression=zlib0`)
Line 135: Line 135:
|align="right" | 04m 36s
|align="right" | 04m 36s
|align="right" | 06m 54s
|align="right" | 06m 54s
|align="right" | 1.5Go
|align="right" | 1.5GB
|align="right" | 3.7Mo/s
|align="right" | 3.7MB/s
|align="right" | 16m 28s
|align="right" | 16m 28s
|align="right" | 01m 01s
|align="right" | 01m 01s
Line 152: Line 152:
| phase 1
| phase 1
| phase 2
| phase 2
| Data send
| Data sent
| Throughput (phase 2)
| Throughput (phase 2)
| Duration on server
| Duration on server
| phase 3
| phase 3
|-
|-
| 227.44Go
| 227.44GB
| Enabled<<[[FootNote]](Note: default value for librsync)>>
| Enabled<<[[FootNote]](Note: default value for librsync)>>
| AES128-SHA
| AES128-SHA
Line 165: Line 165:
|align="right" | 08m 51s
|align="right" | 08m 51s
|align="right" | 28h 08m 50s
|align="right" | 28h 08m 50s
|align="right" | 227.44Go
|align="right" | 227.44GB
|align="right" | 2.29Mo/s
|align="right" | 2.29MB/s
|align="right" | 09m 37s
|align="right" | 09m 37s
|align="right" | 01m 21s
|align="right" | 01m 21s
|-
|-
| 227.45Go
| 227.45GB
| Enabled
| Enabled
| DHE-RSA-AES256-GCM-SHA384
| DHE-RSA-AES256-GCM-SHA384
Line 178: Line 178:
|align="right" | 06m 28s
|align="right" | 06m 28s
|align="right" | 44h 56m 27s
|align="right" | 44h 56m 27s
|align="right" | 176.91Go
|align="right" | 176.91GB
|align="right" | 1.11Mo/s
|align="right" | 1.11MB/s
|align="right" | 12m 02s
|align="right" | 12m 02s
|align="right" | 03m 08s
|align="right" | 03m 08s
|-
|-
| 227.48Go
| 227.48GB
| Enabled
| Enabled
| DHE-RSA-AES256-GCM-SHA384
| DHE-RSA-AES256-GCM-SHA384
Line 191: Line 191:
|align="right" | 06m 26s
|align="right" | 06m 26s
|align="right" | 44h 51m 09s
|align="right" | 44h 51m 09s
|align="right" | 176.94Go
|align="right" | 176.94GB
|align="right" | 1.12Mo/s
|align="right" | 1.12MB/s
|align="right" | 11m 32s
|align="right" | 11m 32s
|align="right" | 03m 06s
|align="right" | 03m 06s
|-
|-
| 227.49Go
| 227.49GB
| Enabled
| Enabled
| DHE-RSA-AES256-GCM-SHA384
| DHE-RSA-AES256-GCM-SHA384
Line 204: Line 204:
|align="right" | 09m 39s
|align="right" | 09m 39s
|align="right" | 28h 09m 37s
|align="right" | 28h 09m 37s
|align="right" | 227.49Go
|align="right" | 227.49GB
|align="right" | 2.29Mo/s
|align="right" | 2.29MB/s
|align="right" | 09m 37s
|align="right" | 09m 37s
|align="right" | 01m 24s
|align="right" | 01m 24s
|-
|-
| 227.52Go
| 227.52GB
| Disabled<<[[FootNote]](Note: librsync=0)>>
| Disabled<<[[FootNote]](Note: librsync=0)>>
| DHE-RSA-AES256-GCM-SHA384
| DHE-RSA-AES256-GCM-SHA384
Line 217: Line 217:
|align="right" | 11m 30s
|align="right" | 11m 30s
|align="right" | 28h 28m 42s
|align="right" | 28h 28m 42s
|align="right" | 227.52Go
|align="right" | 227.52GB
|align="right" | 2.27Mo/s
|align="right" | 2.27MB/s
|align="right" | 09m 43s
|align="right" | 09m 43s
|align="right" | 01m 23s
|align="right" | 01m 23s
|-
|-
| 227.63Go
| 227.63GB
| Disabled<<[[FootNote]](Note: librsync=0)>>
| Disabled<<[[FootNote]](Note: librsync=0)>>
| DHE-RSA-AES256-GCM-SHA384
| DHE-RSA-AES256-GCM-SHA384
Line 230: Line 230:
|align="right" | 10m 17s
|align="right" | 10m 17s
|align="right" | 28h 00m 29s
|align="right" | 28h 00m 29s
|align="right" | 227.63Go
|align="right" | 227.63GB
|align="right" | 2.31Mo/s
|align="right" | 2.31MB/s
|align="right" | 09m 58s
|align="right" | 09m 58s
|align="right" | 01m 26s
|align="right" | 01m 26s
|-
|-
| 227.65Go
| 227.65GB
| Disabled<<[[FootNote]](Note: librsync=0)>>
| Disabled<<[[FootNote]](Note: librsync=0)>>
| DHE-RSA-AES256-GCM-SHA384
| DHE-RSA-AES256-GCM-SHA384
Line 243: Line 243:
|align="right" | 10m 38s
|align="right" | 10m 38s
|align="right" | 17h 45m 51s
|align="right" | 17h 45m 51s
|align="right" | 227.65Go
|align="right" | 227.65GB
|align="right" | 3.64Mo/s
|align="right" | 3.64MB/s
|align="right" | 09m 58s
|align="right" | 09m 58s
|align="right" | 01m 25s
|align="right" | 01m 25s
Line 251: Line 251:
=== Logs ===
=== Logs ===


'''Client1 full backup: 214.27Go'''
'''Client1 full backup: 214.27GB'''


<pre><nowiki>
<pre><nowiki>
Line 289: Line 289:




'''Client 1 full backup 212.47Go'''
'''Client 1 full backup 212.47GB'''


<pre><nowiki>
<pre><nowiki>
Line 327: Line 327:




'''Client 1 full backup 213.99Go'''
'''Client 1 full backup 213.99GB'''


<pre><nowiki>
<pre><nowiki>
Line 365: Line 365:




'''Client 2 full backup 227.44Go'''
'''Client 2 full backup 227.44GB'''


<pre><nowiki>
<pre><nowiki>
Line 403: Line 403:




'''Client 2 full backup 227.45Go'''
'''Client 2 full backup 227.45GB'''


<pre><nowiki>
<pre><nowiki>
Line 441: Line 441:




'''Client 2 full backup 227.49Go'''
'''Client 2 full backup 227.49GB'''


<pre><nowiki>
<pre><nowiki>
Line 479: Line 479:




'''Client 2 full backup 227.52Go'''
'''Client 2 full backup 227.52GB'''


<pre><nowiki>
<pre><nowiki>
Line 518: Line 518:




'''Client 2 full backup 227.63Go'''
'''Client 2 full backup 227.63GB'''


<pre><nowiki>
<pre><nowiki>
Line 556: Line 556:




'''Client 2 full backup 227.65Go'''
'''Client 2 full backup 227.65GB'''


<pre><nowiki>
<pre><nowiki>
Line 608: Line 608:
| Throughput (phase 2)
| Throughput (phase 2)
|-
|-
| First backup: 210Go
| First backup: 210GB
| None (`compression=gzip0`)
| None (`compression=gzip0`)
| (default parameter `ssl_compression` not set)
| (default parameter `ssl_compression` not set)
| 37h 09m 30s <span style="font-size: smaller">(18m 56s/36h 50m 34s)</span>
| 37h 09m 30s <span style="font-size: smaller">(18m 56s/36h 50m 34s)</span>
|align="right" | 1.6 Mo/s
|align="right" | 1.6 MB/s
|-
|-
| Second backup: 34.33Mo added
| Second backup: 34.33MB added
| None (`compression=gzip0`)
| None (`compression=gzip0`)
| (default parameter `ssl_compression` not set)
| (default parameter `ssl_compression` not set)
| 00h 18m 10s <span style="font-size: smaller">(16m 45s/1m 25s)</span>
| 00h 18m 10s <span style="font-size: smaller">(16m 45s/1m 25s)</span>
|align="right" | 0.4 Mo/s
|align="right" | 0.4 MB/s
|-
|-
| Third backup: 4.14Go (mp4 files) added
| Third backup: 4.14GB (mp4 files) added
| None (`compression=gzip0`)
| None (`compression=gzip0`)
| (default parameter `ssl_compression` not set)
| (default parameter `ssl_compression` not set)
| 01h 19m 54s <span style="font-size: smaller">(11m 56s/1h 07m 58s)</span>
| 01h 19m 54s <span style="font-size: smaller">(11m 56s/1h 07m 58s)</span>
|align="right" | 1.03 Mo/s
|align="right" | 1.03 MB/s
|-
|-
| Fourth backup: 4.24Go (mp4 files) added
| Fourth backup: 4.24GB (mp4 files) added
| None (`compression=gzip0`)
| None (`compression=gzip0`)
| `ssl_compression=gzip0`
| `ssl_compression=gzip0`
| 00h 58m 16s <span style="font-size: smaller">(16m 39s/41m 37s)</span>
| 00h 58m 16s <span style="font-size: smaller">(16m 39s/41m 37s)</span>
|align="right" | 1.73 Mo/s
|align="right" | 1.73 MB/s
|}
|}



Revision as of 16:06, 16 October 2014

Burp tests

Description

Version: 1.3.48 (debian package 1.3.48-4~bpo70+1)

Hardware configuration

Type CPU Disk RAM
Server Marvell Armada 370 (ReadyNAS 104) 2*WDC WD20EFRX-68EUZN0 (soft raid1 using mdadm) 512MB
Client1 VIA Nano L2007 1600MHz 2*WDC WD20EFRX-68EUZN0 (soft raid1 using mdadm) 4GB
Client2 VIA Nano L2007 1600MHz 2*WDC WD20EFRX-68EUZN0 (soft raid1 using mdadm) 4GB

All hosts have gigabit NICs with properly cabled CAT 6 cables and all are connected to a Gigabit switch (Netgear GS108E).

Ethernet controllers and modules:

  • client: `Realtek Semiconductor Co., Ltd. RTL8111/8168B PCI Express Gigabit Ethernet controller (rev 06)` (controller), `r8168` (module)
  • server: `Marvell 88E1318 Gigabit Ethernet PHY` (controller), `pegasus` (module)

Performances

Network

  • `iperf` measures 890 Mbits/s between server and client.
  • Copying 244.31GB grom client to backup server using rsync took 36h 21m (`sent 189.41Gbytes - received 25.48Mbytes - 1.45Mbytes/sec`).
  • Copying 212GB from client1 (VIA PadLock usable by OpenSSL) to backup server using `scp` (default cipher specification and MAC algorithm) took 8h 1m 48s (7.50MB/s).

Encryption

  • SCP transfers of 698MB from the client1 (VIA PadLock usable by OpenSSL) to the server using different cipher and MAC algorithm (`aes128-ctr` and `hmac-md5` are the default settings on `Debian Wheezy`):
|| cipher specification || MAC algorithm || Throughput || Duration ||
|| aes128-ctr || hmac-md5         ||  5.9MB/s || 01:59 ||
|| aes128-ctr || hmac-sha2-256    || 10.7MB/s || 01:05 ||
|| aes128-ctr || hmac-sha2-256-96 || 10.7MB/s || 01:05 ||
|| aes128-ctr || hmac-sha1        || 11.3MB/s || 01:02 ||
|| aes128-ctr || hmac-sha1-96     || 11.3MB/s || 01:02 ||
|| aes128-ctr || hmac-sha2-512    ||  8.3MB/s || 01:24 ||
|| aes192-ctr || hmac-sha2-256    ||  9.4MB/s || 01:14 ||
|| aes256-ctr || hmac-sha2-256    ||  8.4MB/s || 01:23 ||
|| aes128-cbc || hmac-sha1        || 14.0MB/s || 00:50 ||
|| aes128-cbc || hmac-sha2-256    || 11.8MB/s || 00:59 ||
|| aes192-cbc || hmac-sha2-256    || 11.5MB/s || 01:01 ||

Test using client1

Description Compression (`compression`) <<FootNote(Note: Extensions which are not compressed: bz, bz2, zip, gz, webm, mov, png, jpg, jpeg, rar, m4v, avi, xv, jar, tgz, mp4, mkv, ogg, flv)>> SSL compression (`ssl_compression`) Duration on client phase 1 phase 2 Data sent Throughput (phase 2) Duration on server phase 3
full backup: 209.17GB gzip9 (default value) zlib5 (default value) 34h 17m 34s 07m 18s 34h 10m 18s 167.4GB 1.4MB/s 06m 22s 02m 22s
full backup: 209.17GB None (`compression=gzip0`) zlib5 (default value) 14h 49m 29s 03m 57s 14h 45m 32s 209.17GB 4.0MB/s 07m 04s 01m 09s
full backup: 214.27GB None (`compression=gzip0`) zlib5 (default value) 30h 05m 44s 7m 45s 29h 57m 59s 214.27GB 2.0MB/s 06m 56s 01m 01s
full backup: 212.47GB None (`compression=gzip0`) None (`ssl_compression=zlib0`) 24h 19m 53s 04m 36s 24h 15m 17s 212.47GB 2.5MB/s 06m 57s 54s
full backup: 213.99GB None (`compression=gzip0`) None (`ssl_compression=zlib0`) 30h 00m 45s 05m 47s 29h 54m 58s 213.99GB 2.0MB/s 06m 59s 01m 05s
incremental backup: 1.5GB added None (`compression=gzip0`) None (`ssl_compression=zlib0`) 11m 30s 04m 36s 06m 54s 1.5GB 3.7MB/s 16m 28s 01m 01s

Test using client2: full backups

Description librsync SSL algo `compression` <<FootNote(Note: Extensions which are not compressed: bz, bz2, zip, gz, webm, mov, png, jpg, jpeg, rar, m4v, avi, xv, jar, tgz, mp4, mkv, ogg, flv)>> `ssl_compression` Duration on client phase 1 phase 2 Data sent Throughput (phase 2) Duration on server phase 3
227.44GB Enabled<<FootNote(Note: default value for librsync)>> AES128-SHA gzip0 zlib5 (default value) 28h 17m 41s 08m 51s 28h 08m 50s 227.44GB 2.29MB/s 09m 37s 01m 21s
227.45GB Enabled DHE-RSA-AES256-GCM-SHA384 gzip9 (default value) zlib5 (default value) 45h 02m 55s 06m 28s 44h 56m 27s 176.91GB 1.11MB/s 12m 02s 03m 08s
227.48GB Enabled DHE-RSA-AES256-GCM-SHA384 gzip9 (default value) zlib0 44h 57m 35s 06m 26s 44h 51m 09s 176.94GB 1.12MB/s 11m 32s 03m 06s
227.49GB Enabled DHE-RSA-AES256-GCM-SHA384 gzip0 zlib5 (default value) 28h 19m 06s 09m 39s 28h 09m 37s 227.49GB 2.29MB/s 09m 37s 01m 24s
227.52GB Disabled<<FootNote(Note: librsync=0)>> DHE-RSA-AES256-GCM-SHA384 gzip0 zlib5 (default value) 28h 40m 12s 11m 30s 28h 28m 42s 227.52GB 2.27MB/s 09m 43s 01m 23s
227.63GB Disabled<<FootNote(Note: librsync=0)>> DHE-RSA-AES256-GCM-SHA384 gzip0 zlib5 (default value) 28h 10m 46s 10m 17s 28h 00m 29s 227.63GB 2.31MB/s 09m 58s 01m 26s
227.65GB Disabled<<FootNote(Note: librsync=0)>> DHE-RSA-AES256-GCM-SHA384 gzip0 zlib5 (default value) 17h 56m 29s 10m 38s 17h 45m 51s 227.65GB 3.64MB/s 09m 58s 01m 25s

Logs

Client1 full backup: 214.27GB

2014-07-19 18:32:40: burp[9606] Client version: 1.3.48
2014-07-19 18:32:40: burp[9606] Begin phase1 (file system scan)
2014-07-19 18:40:25: burp[9606] End phase1 (file system scan)
2014-07-19 18:40:25: burp[9606] Begin phase2 (receive file data)
2014-07-21 00:38:24: burp[9606] End phase2 (receive file data)
2014-07-21 00:38:24: burp[9606] Begin phase3 (merge manifests)
2014-07-21 00:39:25: burp[9606] End phase3 (merge manifests)
2014-07-21 00:39:25: burp[9606] Backup ending - disconnect from client.
2014-07-21 00:39:25: burp[9606] Begin phase4 (shuffle files)
2014-07-21 00:39:25: burp[9606] Doing the atomic data jiggle...
--------------------------------------------------------------------------------
Start time: 2014-07-19 18:32:39
  End time: 2014-07-21 00:45:20
Time taken: 01:06:12:41
                         New   Changed Unchanged   Deleted     Total |  Scanned
                   ------------------------------------------------------------
            Files:    998898         0         0         0    998898 |   998898
      Directories:     74226         0         0         0     74226 |    74226
       Soft links:      4531         0         0         0      4531 |     4531
       Hard links:        16         0         0         0        16 |       16
      Grand total:   1077671         0         0         0   1077671 |  1077671
                   ------------------------------------------------------------

             Warnings:             0

      Bytes estimated:   230074235176 (214.27 GB)
      Bytes in backup:   230074235177 (214.27 GB)
       Bytes received:   230074235177 (214.27 GB)
           Bytes sent:             0
--------------------------------------------------------------------------------
2014-07-21 00:45:20: burp[9606] Backup completed.
2014-07-21 00:45:20: burp[9606] End phase4 (shuffle files)


Client 1 full backup 212.47GB

2014-07-14 23:49:34: burp[5369] Client version: 1.3.48
2014-07-14 23:49:34: burp[5369] Begin phase1 (file system scan)
2014-07-14 23:54:10: burp[5369] End phase1 (file system scan)
2014-07-14 23:54:10: burp[5369] Begin phase2 (receive file data)
2014-07-16 01:09:27: burp[5369] End phase2 (receive file data)
2014-07-16 01:09:27: burp[5369] Begin phase3 (merge manifests)
2014-07-16 01:10:31: burp[5369] End phase3 (merge manifests)
2014-07-16 01:10:31: burp[5369] Backup ending - disconnect from client.
2014-07-16 01:10:31: burp[5369] Begin phase4 (shuffle files)
2014-07-16 01:10:31: burp[5369] Doing the atomic data jiggle...
--------------------------------------------------------------------------------
Start time: 2014-07-14 23:49:26
  End time: 2014-07-16 01:16:34
Time taken: 01:01:27:08
                         New   Changed Unchanged   Deleted     Total |  Scanned
                   ------------------------------------------------------------
            Files:    998898         0         0         0    998898 |   998898
      Directories:     74226         0         0         0     74226 |    74226
       Soft links:      4531         0         0         0      4531 |     4531
       Hard links:        16         0         0         0        16 |       16
      Grand total:   1077671         0         0         0   1077671 |  1077671
                   ------------------------------------------------------------

             Warnings:             0

      Bytes estimated:   228124139118 (212.46 GB)
      Bytes in backup:   228135345774 (212.47 GB)
       Bytes received:   228135345774 (212.47 GB)
           Bytes sent:             0
--------------------------------------------------------------------------------
2014-07-16 01:16:34: burp[5369] Backup completed.
2014-07-16 01:16:34: burp[5369] End phase4 (shuffle files)


Client 1 full backup 213.99GB

2014-07-17 01:47:37: burp[4654] Client version: 1.3.48
2014-07-17 01:47:37: burp[4654] Begin phase1 (file system scan)
2014-07-17 01:53:24: burp[4654] End phase1 (file system scan)
2014-07-17 01:53:24: burp[4654] Begin phase2 (receive file data)
2014-07-18 07:48:22: burp[4654] End phase2 (receive file data)
2014-07-18 07:48:22: burp[4654] Begin phase3 (merge manifests)
2014-07-18 07:49:27: burp[4654] End phase3 (merge manifests)
2014-07-18 07:49:27: burp[4654] Backup ending - disconnect from client.
2014-07-18 07:49:27: burp[4654] Begin phase4 (shuffle files)
2014-07-18 07:49:27: burp[4654] Doing the atomic data jiggle...
--------------------------------------------------------------------------------
Start time: 2014-07-17 01:47:36
  End time: 2014-07-18 07:55:21
Time taken: 01:06:07:45
                         New   Changed Unchanged   Deleted     Total |  Scanned
                   ------------------------------------------------------------
            Files:    998898         0         0         0    998898 |   998898
      Directories:     74226         0         0         0     74226 |    74226
       Soft links:      4531         0         0         0      4531 |     4531
       Hard links:        16         0         0         0        16 |       16
      Grand total:   1077671         0         0         0   1077671 |  1077671
                   ------------------------------------------------------------

             Warnings:             0

      Bytes estimated:   228149960334 (212.48 GB)
      Bytes in backup:   229772435086 (213.99 GB)
       Bytes received:   229772435086 (213.99 GB)
           Bytes sent:             0
--------------------------------------------------------------------------------
2014-07-18 07:55:21: burp[4654] Backup completed.
2014-07-18 07:55:21: burp[4654] End phase4 (shuffle files)


Client 2 full backup 227.44GB

2014-08-23 15:18:35: burp[4691] Client version: 1.3.48
2014-08-23 15:18:35: burp[4691] Begin phase1 (file system scan)
2014-08-23 15:27:26: burp[4691] End phase1 (file system scan)
2014-08-23 15:27:26: burp[4691] Begin phase2 (receive file data)
2014-08-24 19:36:16: burp[4691] End phase2 (receive file data)
2014-08-24 19:36:16: burp[4691] Begin phase3 (merge manifests)
2014-08-24 19:37:37: burp[4691] End phase3 (merge manifests)
2014-08-24 19:37:37: burp[4691] Backup ending - disconnect from client.
2014-08-24 19:37:37: burp[4691] Begin phase4 (shuffle files)
2014-08-24 19:37:37: burp[4691] Doing the atomic data jiggle...
--------------------------------------------------------------------------------
Start time: 2014-08-23 15:18:34
  End time: 2014-08-24 19:45:53
Time taken: 01:04:27:19
                         New   Changed Unchanged   Deleted     Total |  Scanned
                   ------------------------------------------------------------
            Files:   1318031         0         0         0   1318031 |  1318286
      Directories:    104066         0         0         0    104066 |   104066
       Soft links:      4586         0         0         0      4586 |     4586
       Hard links:        99         0         0         0        99 |       99
      Grand total:   1426782         0         0         0   1426782 |  1427037
                   ------------------------------------------------------------

             Warnings:           255

      Bytes estimated:   244227872076 (227.45 GB)
      Bytes in backup:   244212565354 (227.44 GB)
       Bytes received:   244212565354 (227.44 GB)
           Bytes sent:             0
--------------------------------------------------------------------------------
2014-08-24 19:45:53: burp[4691] Backup completed.
2014-08-24 19:45:53: burp[4691] End phase4 (shuffle files)


Client 2 full backup 227.45GB

2014-08-25 01:23:42: burp[9427] Client version: 1.3.48
2014-08-25 01:23:42: burp[9427] Begin phase1 (file system scan)
2014-08-25 01:30:10: burp[9427] End phase1 (file system scan)
2014-08-25 01:30:10: burp[9427] Begin phase2 (receive file data)
2014-08-26 22:26:37: burp[9427] End phase2 (receive file data)
2014-08-26 22:26:37: burp[9427] Begin phase3 (merge manifests)
2014-08-26 22:29:45: burp[9427] End phase3 (merge manifests)
2014-08-26 22:29:45: burp[9427] Backup ending - disconnect from client.
2014-08-26 22:29:45: burp[9427] Begin phase4 (shuffle files)
2014-08-26 22:29:45: burp[9427] Doing the atomic data jiggle...
--------------------------------------------------------------------------------
Start time: 2014-08-25 01:23:42
  End time: 2014-08-26 22:38:09
Time taken: 01:21:14:27
                         New   Changed Unchanged   Deleted     Total |  Scanned
                   ------------------------------------------------------------
            Files:   1318211         0         0         0   1318211 |  1318211
      Directories:    104070         0         0         0    104070 |   104070
       Soft links:      4586         0         0         0      4586 |     4586
       Hard links:        99         0         0         0        99 |       99
      Grand total:   1426966         0         0         0   1426966 |  1426966
                   ------------------------------------------------------------

             Warnings:             0

      Bytes estimated:   244219700992 (227.45 GB)
      Bytes in backup:   244219700992 (227.45 GB)
       Bytes received:   189963848136 (176.92 GB)
           Bytes sent:             0
--------------------------------------------------------------------------------
2014-08-26 22:38:09: burp[9427] Backup completed.
2014-08-26 22:38:09: burp[9427] End phase4 (shuffle files)


Client 2 full backup 227.49GB

2014-08-30 17:26:31: burp[31130] Client version: 1.3.48
2014-08-30 17:26:31: burp[31130] Begin phase1 (file system scan)
2014-08-30 17:36:10: burp[31130] End phase1 (file system scan)
2014-08-30 17:36:10: burp[31130] Begin phase2 (receive file data)
2014-08-31 21:46:47: burp[31130] End phase2 (receive file data)
2014-08-31 21:46:47: burp[31130] Begin phase3 (merge manifests)
2014-08-31 21:48:11: burp[31130] End phase3 (merge manifests)
2014-08-31 21:48:11: burp[31130] Backup ending - disconnect from client.
2014-08-31 21:48:11: burp[31130] Begin phase4 (shuffle files)
2014-08-31 21:48:11: burp[31130] Doing the atomic data jiggle...
--------------------------------------------------------------------------------
Start time: 2014-08-30 17:26:30
  End time: 2014-08-31 21:56:32
Time taken: 01:04:30:02
                         New   Changed Unchanged   Deleted     Total |  Scanned
                   ------------------------------------------------------------
            Files:   1320475         0         0         0   1320475 |  1320484
      Directories:    104340         0         0         0    104340 |   104340
       Soft links:      4588         0         0         0      4588 |     4588
       Hard links:        99         0         0         0        99 |       99
      Grand total:   1429502         0         0         0   1429502 |  1429511
                   ------------------------------------------------------------

             Warnings:             9

      Bytes estimated:   244267585965 (227.49 GB)
      Bytes in backup:   244267822787 (227.49 GB)
       Bytes received:   244267822787 (227.49 GB)
           Bytes sent:             0
--------------------------------------------------------------------------------
2014-08-31 21:56:32: burp[31130] Backup completed.
2014-08-31 21:56:32: burp[31130] End phase4 (shuffle files)


Client 2 full backup 227.52GB

2014-09-07 00:42:33: burp[22795] Client version: 1.3.48
2014-09-07 00:42:33: burp[22795] Begin phase1 (file system scan)
2014-09-07 00:54:03: burp[22795] End phase1 (file system scan)
2014-09-07 00:54:03: burp[22795] Begin phase2 (receive file data)
2014-09-08 05:22:45: burp[22795] End phase2 (receive file data)
2014-09-08 05:22:45: burp[22795] Begin phase3 (merge manifests)
2014-09-08 05:24:08: burp[22795] End phase3 (merge manifests)
2014-09-08 05:24:08: burp[22795] Backup ending - disconnect from client.
2014-09-08 05:24:08: burp[22795] Begin phase4 (shuffle files)
2014-09-08 05:24:08: burp[22795] Doing the atomic data jiggle...
--------------------------------------------------------------------------------
Start time: 2014-09-07 00:42:32
  End time: 2014-09-08 05:32:25
Time taken: 01:04:49:53
                         New   Changed Unchanged   Deleted     Total |  Scanned
                   ------------------------------------------------------------
            Files:   1322008         0         0         0   1322008 |  1322008
      Directories:    104595         0         0         0    104595 |   104595
       Soft links:      4588         0         0         0      4588 |     4588
       Hard links:        99         0         0         0        99 |       99
      Grand total:   1431290         0         0         0   1431290 |  1431290
                   ------------------------------------------------------------

             Warnings:             0

      Bytes estimated:   244299698290 (227.52 GB)
      Bytes in backup:   244299767377 (227.52 GB)
       Bytes received:   244299767377 (227.52 GB)
           Bytes sent:             0
--------------------------------------------------------------------------------
2014-09-08 05:32:26: burp[22795] Backup completed.
2014-09-08 05:32:26: burp[22795] End phase4 (shuffle files)


Client 2 full backup 227.63GB

2014-09-16 01:07:22: burp[3507] Client version: 1.3.48
2014-09-16 01:07:22: burp[3507] Begin phase1 (file system scan)
2014-09-16 01:17:39: burp[3507] End phase1 (file system scan)
2014-09-16 01:17:39: burp[3507] Begin phase2 (receive file data)
2014-09-17 05:18:08: burp[3507] End phase2 (receive file data)
2014-09-17 05:18:08: burp[3507] Begin phase3 (merge manifests)
2014-09-17 05:19:34: burp[3507] End phase3 (merge manifests)
2014-09-17 05:19:34: burp[3507] Backup ending - disconnect from client.
2014-09-17 05:19:34: burp[3507] Begin phase4 (shuffle files)
2014-09-17 05:19:34: burp[3507] Doing the atomic data jiggle...
--------------------------------------------------------------------------------
Start time: 2014-09-16 01:07:20
  End time: 2014-09-17 05:28:06
Time taken: 01:04:20:46
                         New   Changed Unchanged   Deleted     Total |  Scanned
                   ------------------------------------------------------------
            Files:   1334116         0         0         0   1334116 |  1334117
      Directories:    110080         0         0         0    110080 |   110080
       Soft links:      5461         0         0         0      5461 |     5461
       Hard links:        99         0         0         0        99 |       99
      Grand total:   1449756         0         0         0   1449756 |  1449757
                   ------------------------------------------------------------

             Warnings:             1

      Bytes estimated:   244419793362 (227.63 GB)
      Bytes in backup:   244419785986 (227.63 GB)
       Bytes received:   244419785986 (227.63 GB)
           Bytes sent:             0
--------------------------------------------------------------------------------
2014-09-17 05:28:06: burp[3507] Backup completed.
2014-09-17 05:28:06: burp[3507] End phase4 (shuffle files)


Client 2 full backup 227.65GB


2014-10-15 03:17:06: burp[31867] Client version: 1.3.48
2014-10-15 03:17:06: burp[31867] Begin phase1 (file system scan)
2014-10-15 03:27:44: burp[31867] End phase1 (file system scan)
2014-10-15 03:27:44: burp[31867] Begin phase2 (receive file data)
2014-10-15 21:13:35: burp[31867] End phase2 (receive file data)
2014-10-15 21:13:35: burp[31867] Begin phase3 (merge manifests)
2014-10-15 21:15:00: burp[31867] End phase3 (merge manifests)
2014-10-15 21:15:00: burp[31867] Backup ending - disconnect from client.
2014-10-15 21:15:00: burp[31867] Begin phase4 (shuffle files)
2014-10-15 21:15:00: burp[31867] Doing the atomic data jiggle...
--------------------------------------------------------------------------------
Start time: 2014-10-15 03:17:05
  End time: 2014-10-15 21:23:32
Time taken: 18:06:27
                         New   Changed Unchanged   Deleted     Total |  Scanned
                   ------------------------------------------------------------
            Files:   1334339         0         0         0   1334339 |  1334339
      Directories:    110116         0         0         0    110116 |   110116
       Soft links:      5461         0         0         0      5461 |     5461
       Hard links:        99         0         0         0        99 |       99
      Grand total:   1450015         0         0         0   1450015 |  1450015
                   ------------------------------------------------------------

             Warnings:             0

      Bytes estimated:   244439612263 (227.65 GB)
      Bytes in backup:   244439612178 (227.65 GB)
       Bytes received:   244439612178 (227.65 GB)
           Bytes sent:             0
--------------------------------------------------------------------------------
2014-10-15 21:23:33: burp[31867] Backup completed.
2014-10-15 21:23:33: burp[31867] End phase4 (shuffle files)

Old tests

There was a network configuration problem: `r8169` module was used instead of `r8168` on the client.

Test using client1

Description Compression SSL compression Duration on client (phase 1/phase 2) Throughput (phase 2)
First backup: 210GB None (`compression=gzip0`) (default parameter `ssl_compression` not set) 37h 09m 30s (18m 56s/36h 50m 34s) 1.6 MB/s
Second backup: 34.33MB added None (`compression=gzip0`) (default parameter `ssl_compression` not set) 00h 18m 10s (16m 45s/1m 25s) 0.4 MB/s
Third backup: 4.14GB (mp4 files) added None (`compression=gzip0`) (default parameter `ssl_compression` not set) 01h 19m 54s (11m 56s/1h 07m 58s) 1.03 MB/s
Fourth backup: 4.24GB (mp4 files) added None (`compression=gzip0`) `ssl_compression=gzip0` 00h 58m 16s (16m 39s/41m 37s) 1.73 MB/s

Test using client2

Description Compression SSL compression Duration on client (phase 1/phase 2) Throughput (phase 2)
First backup: None (`compression=gzip0`) (default parameter `ssl_compression` not set) 37h 06m 53s (15m 40s/36h 51m 13s)