Burp: Difference between revisions

From DcSharedWiki
(Add scp throughput using client3 and scp)
(SATA: add version)
Line 11: Line 11:
| Type
| Type
| CPU
| CPU
| SATA link
| Disk
| Disk
| RAM
| RAM
|-
|-
| Server
| Server
| Marvell Armada 370 (ReadyNAS 104)
| Marvell Armada 370 (ReadyNAS 102)
| `SATA link up 6.0 Gbps (SStatus 133 SControl 300)`
| 2*WDC WD20EFRX-68EUZN0 (soft raid1 using mdadm)
| 2*WDC WD20EFRX-68EUZN0 (soft raid1 using mdadm)
| 512MB
| 512MB
Line 21: Line 23:
| Client1
| Client1
| VIA Nano L2007 1600MHz
| VIA Nano L2007 1600MHz
| `SATA link up 1.5 Gbps`
| 2*WDC WD20EFRX-68EUZN0 (soft raid1 using mdadm)
| 2*WDC WD20EFRX-68EUZN0 (soft raid1 using mdadm)
| 4GB
| 4GB
Line 26: Line 29:
| Client2
| Client2
| VIA Nano L2007 1600MHz
| VIA Nano L2007 1600MHz
| `SATA link up 1.5 Gbps`
| 2*WDC WD20EFRX-68EUZN0 (soft raid1 using mdadm)
| 2*WDC WD20EFRX-68EUZN0 (soft raid1 using mdadm)
| 4GB
| 4GB
Line 32: Line 36:
| Intel Core i7-3770
| Intel Core i7-3770
| WDC WD1003FBYX-01Y7B1
| WDC WD1003FBYX-01Y7B1
| `SATA link up 3.0 Gbps`
| 4GB
| 4GB
|}
|}

Revision as of 16:23, 8 January 2015

Burp tests

Description

Version: 1.3.48 (debian package 1.3.48-4~bpo70+1)

Hardware configuration

Type CPU SATA link Disk RAM
Server Marvell Armada 370 (ReadyNAS 102) `SATA link up 6.0 Gbps (SStatus 133 SControl 300)` 2*WDC WD20EFRX-68EUZN0 (soft raid1 using mdadm) 512MB
Client1 VIA Nano L2007 1600MHz `SATA link up 1.5 Gbps` 2*WDC WD20EFRX-68EUZN0 (soft raid1 using mdadm) 4GB
Client2 VIA Nano L2007 1600MHz `SATA link up 1.5 Gbps` 2*WDC WD20EFRX-68EUZN0 (soft raid1 using mdadm) 4GB
Client3 Intel Core i7-3770 WDC WD1003FBYX-01Y7B1 `SATA link up 3.0 Gbps` 4GB

All hosts have gigabit NICs with properly cabled CAT 6 cables and all are connected to a Gigabit switch (Netgear GS108E).

Ethernet controllers and modules:

  • client: `Realtek Semiconductor Co., Ltd. RTL8111/8168B PCI Express Gigabit Ethernet controller (rev 06)` (controller), `r8168` (module)
  • server: `Marvell 88E1318 Gigabit Ethernet PHY` (controller), `pegasus` (module)

Performances

Network

  • `iperf` measures 890 Mbits/s between server and client.
  • Copying 244.31GB grom client to backup server using rsync took 36h 21m (`sent 189.41Gbytes - received 25.48Mbytes - 1.45Mbytes/sec`).
  • Copying 212GB from client1 (VIA PadLock usable by OpenSSL) to backup server using `scp` (cipher specification/MAC algorithm: `aes128-ctr`/`hmac-md5`) took 8h 1m 48s (7.50MB/s).
  • Copying 216GB from client3 to backup server using `scp` (cipher specification/MAC algorithm: `aes128-ctr`/`umac-64@openssh.com`) took 4h 45m 04s (12.93MB/s).

Encryption

  • SCP transfers of 698MB from the client1 (VIA PadLock usable by OpenSSL) to the server using different cipher and MAC algorithm (`aes128-ctr` and `hmac-md5` are the default settings on `Debian Wheezy`):
|| cipher specification || MAC algorithm || Throughput || Duration ||
|| aes128-ctr || hmac-md5         ||  5.9MB/s || 01:59 ||
|| aes128-ctr || hmac-sha2-256    || 10.7MB/s || 01:05 ||
|| aes128-ctr || hmac-sha2-256-96 || 10.7MB/s || 01:05 ||
|| aes128-ctr || hmac-sha1        || 11.3MB/s || 01:02 ||
|| aes128-ctr || hmac-sha1-96     || 11.3MB/s || 01:02 ||
|| aes128-ctr || hmac-sha2-512    ||  8.3MB/s || 01:24 ||
|| aes192-ctr || hmac-sha2-256    ||  9.4MB/s || 01:14 ||
|| aes256-ctr || hmac-sha2-256    ||  8.4MB/s || 01:23 ||
|| aes128-cbc || hmac-sha1        || 14.0MB/s || 00:50 ||
|| aes128-cbc || hmac-sha2-256    || 11.8MB/s || 00:59 ||
|| aes192-cbc || hmac-sha2-256    || 11.5MB/s || 01:01 ||

Test using client1

Description Compression (`compression`) <<FootNote(Note: Extensions which are not compressed: bz, bz2, zip, gz, webm, mov, png, jpg, jpeg, rar, m4v, avi, xv, jar, tgz, mp4, mkv, ogg, flv)>> SSL compression (`ssl_compression`) Duration on client phase 1 phase 2 Data sent Throughput (phase 2) Duration on server phase 3
full backup: 209.17GB gzip9 (default value) zlib5 (default value) 34h 17m 34s 07m 18s 34h 10m 18s 167.4GB 1.4MB/s 06m 22s 02m 22s
full backup: 209.17GB None (`compression=gzip0`) zlib5 (default value) 14h 49m 29s 03m 57s 14h 45m 32s 209.17GB 4.0MB/s 07m 04s 01m 09s
full backup: 214.27GB None (`compression=gzip0`) zlib5 (default value) 30h 05m 44s 7m 45s 29h 57m 59s 214.27GB 2.0MB/s 06m 56s 01m 01s
full backup: 212.47GB None (`compression=gzip0`) None (`ssl_compression=zlib0`) 24h 19m 53s 04m 36s 24h 15m 17s 212.47GB 2.5MB/s 06m 57s 54s
full backup: 213.99GB None (`compression=gzip0`) None (`ssl_compression=zlib0`) 30h 00m 45s 05m 47s 29h 54m 58s 213.99GB 2.0MB/s 06m 59s 01m 05s
incremental backup: 1.5GB added None (`compression=gzip0`) None (`ssl_compression=zlib0`) 11m 30s 04m 36s 06m 54s 1.5GB 3.7MB/s 16m 28s 01m 01s

Test using client2: full backups

Description librsync SSL algo `compression` <<FootNote(Note: Extensions which are not compressed: bz, bz2, zip, gz, webm, mov, png, jpg, jpeg, rar, m4v, avi, xv, jar, tgz, mp4, mkv, ogg, flv)>> `ssl_compression` Duration on client phase 1 phase 2 Data sent Throughput (phase 2) Duration on server phase 3
227.44GB Enabled<<FootNote(Note: default value for librsync)>> AES128-SHA gzip0 zlib5 (default value) 28h 17m 41s 08m 51s 28h 08m 50s 227.44GB 2.29MB/s 09m 37s 01m 21s
227.45GB Enabled DHE-RSA-AES256-GCM-SHA384 gzip9 (default value) zlib5 (default value) 45h 02m 55s 06m 28s 44h 56m 27s 176.91GB 1.11MB/s 12m 02s 03m 08s
227.48GB Enabled DHE-RSA-AES256-GCM-SHA384 gzip9 (default value) zlib0 44h 57m 35s 06m 26s 44h 51m 09s 176.94GB 1.12MB/s 11m 32s 03m 06s
227.49GB Enabled DHE-RSA-AES256-GCM-SHA384 gzip0 zlib5 (default value) 28h 19m 06s 09m 39s 28h 09m 37s 227.49GB 2.29MB/s 09m 37s 01m 24s
227.52GB Disabled<<FootNote(Note: librsync=0)>> DHE-RSA-AES256-GCM-SHA384 gzip0 zlib5 (default value) 28h 40m 12s 11m 30s 28h 28m 42s 227.52GB 2.27MB/s 09m 43s 01m 23s
227.63GB Disabled<<FootNote(Note: librsync=0)>> DHE-RSA-AES256-GCM-SHA384 gzip0 zlib5 (default value) 28h 10m 46s 10m 17s 28h 00m 29s 227.63GB 2.31MB/s 09m 58s 01m 26s
227.65GB Disabled<<FootNote(Note: librsync=0)>> DHE-RSA-AES256-GCM-SHA384 gzip0 zlib5 (default value) 17h 56m 29s 10m 38s 17h 45m 51s 227.65GB 3.64MB/s 09m 58s 01m 25s
227.65GB Disabled<<FootNote(Note: librsync=0)>> DHE-RSA-AES256-GCM-SHA384 gzip0 zlib5 (default value) 17h 24m 56s 03m 50s 17h 21m 06s 227.65GB 3.73MB/s 09m 51s 01m 27s

Test using client3: full backups

Description librsync SSL algo `compression` <<FootNote(Note: Extensions which are not compressed: bz, bz2, zip, gz, webm, mov, png, jpg, jpeg, rar, m4v, avi, xv, jar, tgz, mp4, mkv, ogg, flv)>> `ssl_compression` Duration on client phase 1 phase 2 Data sent Throughput (phase 2) Duration on server phase 3
229.63GB Enabled<<FootNote(Note: default value for librsync)>> DHE-RSA-AES256-GCM-SHA384 gzip0 zlib5 (default value) 06h 44m 39s 00m 15s 06h 44m 24s 229.63GB 9.68MB/s 00m 26s 00m 04s
229.67GB Enabled<<FootNote(Note: default value for librsync)>> DHE-RSA-AES256-GCM-SHA384 gzip9 zlib5 (default value) 06h 59m 43s 00m 14s 06h 59m 29s 227.37GB 9.25MB/s 00m 36s 00m 11s

Test using client2: restore one file

/usr/sbin/burp -c /tmp/burp-client.conf -a r -b 1 -r '^/mnt/data/Documents/file.pdf$' -d /tmp/ -s 4 -b 30


Size of backups: 249GB

Test using client2: incremental backup

Time taken
31m 48s
39m 59s
37m 39s
39m 00s
39m 13s
38m 35s
38m 37s
37m 37s
37m 52s
42m 15s
37m 50s
41m 40s
43m 46s
42m 56s
42m 16s
42m 27s
43m 02s
42m 18s
44m 41s
41m 15s
38m 30s
40m 16s
39m 40s
41m 17s
39m 15s
40m 42s

Logs

Client1 full backup: 214.27GB 

2014-07-19 18:32:40: burp[9606] Client version: 1.3.48
2014-07-19 18:32:40: burp[9606] Begin phase1 (file system scan)
2014-07-19 18:40:25: burp[9606] End phase1 (file system scan)
2014-07-19 18:40:25: burp[9606] Begin phase2 (receive file data)
2014-07-21 00:38:24: burp[9606] End phase2 (receive file data)
2014-07-21 00:38:24: burp[9606] Begin phase3 (merge manifests)
2014-07-21 00:39:25: burp[9606] End phase3 (merge manifests)
2014-07-21 00:39:25: burp[9606] Backup ending - disconnect from client.
2014-07-21 00:39:25: burp[9606] Begin phase4 (shuffle files)
2014-07-21 00:39:25: burp[9606] Doing the atomic data jiggle...
--------------------------------------------------------------------------------
Start time: 2014-07-19 18:32:39
  End time: 2014-07-21 00:45:20
Time taken: 01:06:12:41
                         New   Changed Unchanged   Deleted     Total |  Scanned
                   ------------------------------------------------------------
            Files:    998898         0         0         0    998898 |   998898
      Directories:     74226         0         0         0     74226 |    74226
       Soft links:      4531         0         0         0      4531 |     4531
       Hard links:        16         0         0         0        16 |       16
      Grand total:   1077671         0         0         0   1077671 |  1077671
                   ------------------------------------------------------------

             Warnings:             0

      Bytes estimated:   230074235176 (214.27 GB)
      Bytes in backup:   230074235177 (214.27 GB)
       Bytes received:   230074235177 (214.27 GB)
           Bytes sent:             0
--------------------------------------------------------------------------------
2014-07-21 00:45:20: burp[9606] Backup completed.
2014-07-21 00:45:20: burp[9606] End phase4 (shuffle files)


Client 1 full backup 212.47GB 

2014-07-14 23:49:34: burp[5369] Client version: 1.3.48
2014-07-14 23:49:34: burp[5369] Begin phase1 (file system scan)
2014-07-14 23:54:10: burp[5369] End phase1 (file system scan)
2014-07-14 23:54:10: burp[5369] Begin phase2 (receive file data)
2014-07-16 01:09:27: burp[5369] End phase2 (receive file data)
2014-07-16 01:09:27: burp[5369] Begin phase3 (merge manifests)
2014-07-16 01:10:31: burp[5369] End phase3 (merge manifests)
2014-07-16 01:10:31: burp[5369] Backup ending - disconnect from client.
2014-07-16 01:10:31: burp[5369] Begin phase4 (shuffle files)
2014-07-16 01:10:31: burp[5369] Doing the atomic data jiggle...
--------------------------------------------------------------------------------
Start time: 2014-07-14 23:49:26
  End time: 2014-07-16 01:16:34
Time taken: 01:01:27:08
                         New   Changed Unchanged   Deleted     Total |  Scanned
                   ------------------------------------------------------------
            Files:    998898         0         0         0    998898 |   998898
      Directories:     74226         0         0         0     74226 |    74226
       Soft links:      4531         0         0         0      4531 |     4531
       Hard links:        16         0         0         0        16 |       16
      Grand total:   1077671         0         0         0   1077671 |  1077671
                   ------------------------------------------------------------

             Warnings:             0

      Bytes estimated:   228124139118 (212.46 GB)
      Bytes in backup:   228135345774 (212.47 GB)
       Bytes received:   228135345774 (212.47 GB)
           Bytes sent:             0
--------------------------------------------------------------------------------
2014-07-16 01:16:34: burp[5369] Backup completed.
2014-07-16 01:16:34: burp[5369] End phase4 (shuffle files)


Client 1 full backup 213.99GB 

2014-07-17 01:47:37: burp[4654] Client version: 1.3.48
2014-07-17 01:47:37: burp[4654] Begin phase1 (file system scan)
2014-07-17 01:53:24: burp[4654] End phase1 (file system scan)
2014-07-17 01:53:24: burp[4654] Begin phase2 (receive file data)
2014-07-18 07:48:22: burp[4654] End phase2 (receive file data)
2014-07-18 07:48:22: burp[4654] Begin phase3 (merge manifests)
2014-07-18 07:49:27: burp[4654] End phase3 (merge manifests)
2014-07-18 07:49:27: burp[4654] Backup ending - disconnect from client.
2014-07-18 07:49:27: burp[4654] Begin phase4 (shuffle files)
2014-07-18 07:49:27: burp[4654] Doing the atomic data jiggle...
--------------------------------------------------------------------------------
Start time: 2014-07-17 01:47:36
  End time: 2014-07-18 07:55:21
Time taken: 01:06:07:45
                         New   Changed Unchanged   Deleted     Total |  Scanned
                   ------------------------------------------------------------
            Files:    998898         0         0         0    998898 |   998898
      Directories:     74226         0         0         0     74226 |    74226
       Soft links:      4531         0         0         0      4531 |     4531
       Hard links:        16         0         0         0        16 |       16
      Grand total:   1077671         0         0         0   1077671 |  1077671
                   ------------------------------------------------------------

             Warnings:             0

      Bytes estimated:   228149960334 (212.48 GB)
      Bytes in backup:   229772435086 (213.99 GB)
       Bytes received:   229772435086 (213.99 GB)
           Bytes sent:             0
--------------------------------------------------------------------------------
2014-07-18 07:55:21: burp[4654] Backup completed.
2014-07-18 07:55:21: burp[4654] End phase4 (shuffle files)


Client 2 full backup 227.44GB 

2014-08-23 15:18:35: burp[4691] Client version: 1.3.48
2014-08-23 15:18:35: burp[4691] Begin phase1 (file system scan)
2014-08-23 15:27:26: burp[4691] End phase1 (file system scan)
2014-08-23 15:27:26: burp[4691] Begin phase2 (receive file data)
2014-08-24 19:36:16: burp[4691] End phase2 (receive file data)
2014-08-24 19:36:16: burp[4691] Begin phase3 (merge manifests)
2014-08-24 19:37:37: burp[4691] End phase3 (merge manifests)
2014-08-24 19:37:37: burp[4691] Backup ending - disconnect from client.
2014-08-24 19:37:37: burp[4691] Begin phase4 (shuffle files)
2014-08-24 19:37:37: burp[4691] Doing the atomic data jiggle...
--------------------------------------------------------------------------------
Start time: 2014-08-23 15:18:34
  End time: 2014-08-24 19:45:53
Time taken: 01:04:27:19
                         New   Changed Unchanged   Deleted     Total |  Scanned
                   ------------------------------------------------------------
            Files:   1318031         0         0         0   1318031 |  1318286
      Directories:    104066         0         0         0    104066 |   104066
       Soft links:      4586         0         0         0      4586 |     4586
       Hard links:        99         0         0         0        99 |       99
      Grand total:   1426782         0         0         0   1426782 |  1427037
                   ------------------------------------------------------------

             Warnings:           255

      Bytes estimated:   244227872076 (227.45 GB)
      Bytes in backup:   244212565354 (227.44 GB)
       Bytes received:   244212565354 (227.44 GB)
           Bytes sent:             0
--------------------------------------------------------------------------------
2014-08-24 19:45:53: burp[4691] Backup completed.
2014-08-24 19:45:53: burp[4691] End phase4 (shuffle files)


Client 2 full backup 227.45GB 

2014-08-25 01:23:42: burp[9427] Client version: 1.3.48
2014-08-25 01:23:42: burp[9427] Begin phase1 (file system scan)
2014-08-25 01:30:10: burp[9427] End phase1 (file system scan)
2014-08-25 01:30:10: burp[9427] Begin phase2 (receive file data)
2014-08-26 22:26:37: burp[9427] End phase2 (receive file data)
2014-08-26 22:26:37: burp[9427] Begin phase3 (merge manifests)
2014-08-26 22:29:45: burp[9427] End phase3 (merge manifests)
2014-08-26 22:29:45: burp[9427] Backup ending - disconnect from client.
2014-08-26 22:29:45: burp[9427] Begin phase4 (shuffle files)
2014-08-26 22:29:45: burp[9427] Doing the atomic data jiggle...
--------------------------------------------------------------------------------
Start time: 2014-08-25 01:23:42
  End time: 2014-08-26 22:38:09
Time taken: 01:21:14:27
                         New   Changed Unchanged   Deleted     Total |  Scanned
                   ------------------------------------------------------------
            Files:   1318211         0         0         0   1318211 |  1318211
      Directories:    104070         0         0         0    104070 |   104070
       Soft links:      4586         0         0         0      4586 |     4586
       Hard links:        99         0         0         0        99 |       99
      Grand total:   1426966         0         0         0   1426966 |  1426966
                   ------------------------------------------------------------

             Warnings:             0

      Bytes estimated:   244219700992 (227.45 GB)
      Bytes in backup:   244219700992 (227.45 GB)
       Bytes received:   189963848136 (176.92 GB)
           Bytes sent:             0
--------------------------------------------------------------------------------
2014-08-26 22:38:09: burp[9427] Backup completed.
2014-08-26 22:38:09: burp[9427] End phase4 (shuffle files)


Client 2 full backup 227.49GB 

2014-08-30 17:26:31: burp[31130] Client version: 1.3.48
2014-08-30 17:26:31: burp[31130] Begin phase1 (file system scan)
2014-08-30 17:36:10: burp[31130] End phase1 (file system scan)
2014-08-30 17:36:10: burp[31130] Begin phase2 (receive file data)
2014-08-31 21:46:47: burp[31130] End phase2 (receive file data)
2014-08-31 21:46:47: burp[31130] Begin phase3 (merge manifests)
2014-08-31 21:48:11: burp[31130] End phase3 (merge manifests)
2014-08-31 21:48:11: burp[31130] Backup ending - disconnect from client.
2014-08-31 21:48:11: burp[31130] Begin phase4 (shuffle files)
2014-08-31 21:48:11: burp[31130] Doing the atomic data jiggle...
--------------------------------------------------------------------------------
Start time: 2014-08-30 17:26:30
  End time: 2014-08-31 21:56:32
Time taken: 01:04:30:02
                         New   Changed Unchanged   Deleted     Total |  Scanned
                   ------------------------------------------------------------
            Files:   1320475         0         0         0   1320475 |  1320484
      Directories:    104340         0         0         0    104340 |   104340
       Soft links:      4588         0         0         0      4588 |     4588
       Hard links:        99         0         0         0        99 |       99
      Grand total:   1429502         0         0         0   1429502 |  1429511
                   ------------------------------------------------------------

             Warnings:             9

      Bytes estimated:   244267585965 (227.49 GB)
      Bytes in backup:   244267822787 (227.49 GB)
       Bytes received:   244267822787 (227.49 GB)
           Bytes sent:             0
--------------------------------------------------------------------------------
2014-08-31 21:56:32: burp[31130] Backup completed.
2014-08-31 21:56:32: burp[31130] End phase4 (shuffle files)


Client 2 full backup 227.52GB 

2014-09-07 00:42:33: burp[22795] Client version: 1.3.48
2014-09-07 00:42:33: burp[22795] Begin phase1 (file system scan)
2014-09-07 00:54:03: burp[22795] End phase1 (file system scan)
2014-09-07 00:54:03: burp[22795] Begin phase2 (receive file data)
2014-09-08 05:22:45: burp[22795] End phase2 (receive file data)
2014-09-08 05:22:45: burp[22795] Begin phase3 (merge manifests)
2014-09-08 05:24:08: burp[22795] End phase3 (merge manifests)
2014-09-08 05:24:08: burp[22795] Backup ending - disconnect from client.
2014-09-08 05:24:08: burp[22795] Begin phase4 (shuffle files)
2014-09-08 05:24:08: burp[22795] Doing the atomic data jiggle...
--------------------------------------------------------------------------------
Start time: 2014-09-07 00:42:32
  End time: 2014-09-08 05:32:25
Time taken: 01:04:49:53
                         New   Changed Unchanged   Deleted     Total |  Scanned
                   ------------------------------------------------------------
            Files:   1322008         0         0         0   1322008 |  1322008
      Directories:    104595         0         0         0    104595 |   104595
       Soft links:      4588         0         0         0      4588 |     4588
       Hard links:        99         0         0         0        99 |       99
      Grand total:   1431290         0         0         0   1431290 |  1431290
                   ------------------------------------------------------------

             Warnings:             0

      Bytes estimated:   244299698290 (227.52 GB)
      Bytes in backup:   244299767377 (227.52 GB)
       Bytes received:   244299767377 (227.52 GB)
           Bytes sent:             0
--------------------------------------------------------------------------------
2014-09-08 05:32:26: burp[22795] Backup completed.
2014-09-08 05:32:26: burp[22795] End phase4 (shuffle files)


Client 2 full backup 227.63GB 

2014-09-16 01:07:22: burp[3507] Client version: 1.3.48
2014-09-16 01:07:22: burp[3507] Begin phase1 (file system scan)
2014-09-16 01:17:39: burp[3507] End phase1 (file system scan)
2014-09-16 01:17:39: burp[3507] Begin phase2 (receive file data)
2014-09-17 05:18:08: burp[3507] End phase2 (receive file data)
2014-09-17 05:18:08: burp[3507] Begin phase3 (merge manifests)
2014-09-17 05:19:34: burp[3507] End phase3 (merge manifests)
2014-09-17 05:19:34: burp[3507] Backup ending - disconnect from client.
2014-09-17 05:19:34: burp[3507] Begin phase4 (shuffle files)
2014-09-17 05:19:34: burp[3507] Doing the atomic data jiggle...
--------------------------------------------------------------------------------
Start time: 2014-09-16 01:07:20
  End time: 2014-09-17 05:28:06
Time taken: 01:04:20:46
                         New   Changed Unchanged   Deleted     Total |  Scanned
                   ------------------------------------------------------------
            Files:   1334116         0         0         0   1334116 |  1334117
      Directories:    110080         0         0         0    110080 |   110080
       Soft links:      5461         0         0         0      5461 |     5461
       Hard links:        99         0         0         0        99 |       99
      Grand total:   1449756         0         0         0   1449756 |  1449757
                   ------------------------------------------------------------

             Warnings:             1

      Bytes estimated:   244419793362 (227.63 GB)
      Bytes in backup:   244419785986 (227.63 GB)
       Bytes received:   244419785986 (227.63 GB)
           Bytes sent:             0
--------------------------------------------------------------------------------
2014-09-17 05:28:06: burp[3507] Backup completed.
2014-09-17 05:28:06: burp[3507] End phase4 (shuffle files)


Client 2 full backup 227.65GB 


2014-10-15 03:17:06: burp[31867] Client version: 1.3.48
2014-10-15 03:17:06: burp[31867] Begin phase1 (file system scan)
2014-10-15 03:27:44: burp[31867] End phase1 (file system scan)
2014-10-15 03:27:44: burp[31867] Begin phase2 (receive file data)
2014-10-15 21:13:35: burp[31867] End phase2 (receive file data)
2014-10-15 21:13:35: burp[31867] Begin phase3 (merge manifests)
2014-10-15 21:15:00: burp[31867] End phase3 (merge manifests)
2014-10-15 21:15:00: burp[31867] Backup ending - disconnect from client.
2014-10-15 21:15:00: burp[31867] Begin phase4 (shuffle files)
2014-10-15 21:15:00: burp[31867] Doing the atomic data jiggle...
--------------------------------------------------------------------------------
Start time: 2014-10-15 03:17:05
  End time: 2014-10-15 21:23:32
Time taken: 18:06:27
                         New   Changed Unchanged   Deleted     Total |  Scanned
                   ------------------------------------------------------------
            Files:   1334339         0         0         0   1334339 |  1334339
      Directories:    110116         0         0         0    110116 |   110116
       Soft links:      5461         0         0         0      5461 |     5461
       Hard links:        99         0         0         0        99 |       99
      Grand total:   1450015         0         0         0   1450015 |  1450015
                   ------------------------------------------------------------

             Warnings:             0

      Bytes estimated:   244439612263 (227.65 GB)
      Bytes in backup:   244439612178 (227.65 GB)
       Bytes received:   244439612178 (227.65 GB)
           Bytes sent:             0
--------------------------------------------------------------------------------
2014-10-15 21:23:33: burp[31867] Backup completed.
2014-10-15 21:23:33: burp[31867] End phase4 (shuffle files)


Client 2 full backup 227.65GB 

2014-10-16 12:32:02: burp[3210] Client version: 1.3.48
2014-10-16 12:32:02: burp[3210] Begin phase1 (file system scan)
2014-10-16 12:35:52: burp[3210] End phase1 (file system scan)
2014-10-16 12:35:52: burp[3210] Begin phase2 (receive file data)
2014-10-17 05:57:58: burp[3210] End phase2 (receive file data)
2014-10-17 05:57:58: burp[3210] Begin phase3 (merge manifests)
2014-10-17 05:59:25: burp[3210] End phase3 (merge manifests)
2014-10-17 05:59:25: burp[3210] Backup ending - disconnect from client.
2014-10-17 05:59:25: burp[3210] Begin phase4 (shuffle files)
2014-10-17 05:59:25: burp[3210] Doing the atomic data jiggle...
--------------------------------------------------------------------------------
Start time: 2014-10-16 12:32:00
  End time: 2014-10-17 06:07:50
Time taken: 17:35:50
                         New   Changed Unchanged   Deleted     Total |  Scanned
                   ------------------------------------------------------------
            Files:   1334340         0         0         0   1334340 |  1334340
      Directories:    110116         0         0         0    110116 |   110116
       Soft links:      5461         0         0         0      5461 |     5461
       Hard links:        99         0         0         0        99 |       99
      Grand total:   1450016         0         0         0   1450016 |  1450016
                   ------------------------------------------------------------

             Warnings:             0

      Bytes estimated:   244439619502 (227.65 GB)
      Bytes in backup:   244439631290 (227.65 GB)
       Bytes received:   244439631290 (227.65 GB)
           Bytes sent:             0
--------------------------------------------------------------------------------
2014-10-17 06:07:51: burp[3210] Backup completed.
2014-10-17 06:07:51: burp[3210] End phase4 (shuffle files)


Client 3 full backup 229.63GB 

2014-11-18 03:58:03: burp[4847] Client version: 1.3.48
2014-11-18 03:58:03: burp[4847] Begin phase1 (file system scan)
2014-11-18 03:58:18: burp[4847] End phase1 (file system scan)
2014-11-18 03:58:18: burp[4847] Begin phase2 (receive file data)
2014-11-18 10:42:42: burp[4847] End phase2 (receive file data)
2014-11-18 10:42:42: burp[4847] Begin phase3 (merge manifests)
2014-11-18 10:42:46: burp[4847] End phase3 (merge manifests)
2014-11-18 10:42:46: burp[4847] Backup ending - disconnect from client.
2014-11-18 10:42:46: burp[4847] Begin phase4 (shuffle files)
2014-11-18 10:42:46: burp[4847] Doing the atomic data jiggle...
--------------------------------------------------------------------------------
Start time: 2014-11-18 03:58:03
  End time: 2014-11-18 10:43:08
Time taken: 06:45:05
                         New   Changed Unchanged   Deleted     Total |  Scanned
                   ------------------------------------------------------------
            Files:     69307         0         0         0     69307 |    69307
      Directories:      4273         0         0         0      4273 |     4273
       Soft links:        38         0         0         0        38 |       38
    Special files:        19         0         0         0        19 |       19
      Grand total:     73637         0         0         0     73637 |    73637
                   ------------------------------------------------------------

             Warnings:             0

      Bytes estimated:   246568150349 (229.63 GB)
      Bytes in backup:   246568152212 (229.63 GB)
       Bytes received:   246568152212 (229.63 GB)
           Bytes sent:             0
--------------------------------------------------------------------------------
2014-11-18 10:43:08: burp[4847] Backup completed.
2014-11-18 10:43:08: burp[4847] End phase4 (shuffle files)


Client 3 full backup 229.67GB 

2014-11-18 23:32:58: burp[8331] Client version: 1.3.48
2014-11-18 23:32:58: burp[8331] Begin phase1 (file system scan)
2014-11-18 23:33:12: burp[8331] End phase1 (file system scan)
2014-11-18 23:33:12: burp[8331] Begin phase2 (receive file data)
2014-11-19 06:32:41: burp[8331] End phase2 (receive file data)
2014-11-19 06:32:41: burp[8331] Begin phase3 (merge manifests)
2014-11-19 06:32:52: burp[8331] End phase3 (merge manifests)
2014-11-19 06:32:52: burp[8331] Backup ending - disconnect from client.
2014-11-19 06:32:52: burp[8331] Begin phase4 (shuffle files)
2014-11-19 06:32:52: burp[8331] Doing the atomic data jiggle...
--------------------------------------------------------------------------------
Start time: 2014-11-18 23:32:57
  End time: 2014-11-19 06:33:17
Time taken: 07:00:20
                         New   Changed Unchanged   Deleted     Total |  Scanned
                   ------------------------------------------------------------
            Files:     69554         0         0         0     69554 |    69554
      Directories:      4505         0         0         0      4505 |     4505
       Soft links:        38         0         0         0        38 |       38
    Special files:        19         0         0         0        19 |       19
      Grand total:     74116         0         0         0     74116 |    74116
                   ------------------------------------------------------------

             Warnings:             0

      Bytes estimated:   246605869782 (229.67 GB)
      Bytes in backup:   246605902574 (229.67 GB)
       Bytes received:   244132071693 (227.37 GB)
           Bytes sent:             0
--------------------------------------------------------------------------------
2014-11-19 06:33:17: burp[8331] Backup completed.
2014-11-19 06:33:17: burp[8331] End phase4 (shuffle files)


Old tests

There was a network configuration problem: `r8169` module was used instead of `r8168` on the client.

Test using client1

Description Compression SSL compression Duration on client (phase 1/phase 2) Throughput (phase 2)
First backup: 210GB None (`compression=gzip0`) (default parameter `ssl_compression` not set) 37h 09m 30s (18m 56s/36h 50m 34s) 1.6 MB/s
Second backup: 34.33MB added None (`compression=gzip0`) (default parameter `ssl_compression` not set) 00h 18m 10s (16m 45s/1m 25s) 0.4 MB/s
Third backup: 4.14GB (mp4 files) added None (`compression=gzip0`) (default parameter `ssl_compression` not set) 01h 19m 54s (11m 56s/1h 07m 58s) 1.03 MB/s
Fourth backup: 4.24GB (mp4 files) added None (`compression=gzip0`) `ssl_compression=gzip0` 00h 58m 16s (16m 39s/41m 37s) 1.73 MB/s

Test using client2

Description Compression SSL compression Duration on client (phase 1/phase 2) Throughput (phase 2)
First backup: None (`compression=gzip0`) (default parameter `ssl_compression` not set) 37h 06m 53s (15m 40s/36h 51m 13s)
Retrieved from "https://wiki.duckcorp.org//index.php?title=Burp&oldid=234"