Burp: Difference between revisions

From DcSharedWiki
(Add another run: full backup without compression neither ssl_compression)
(Add new test (full backup))
Line 84: Line 84:
|align="right" | 06m 57s
|align="right" | 06m 57s
|align="right" | 54s
|align="right" | 54s
|-
| full backup: 213.99Go
| None (`compression=gzip0`)
| None (`ssl_compression=zlib0`)
|align="right" | 30h 00m 45s
|align="right" | 05m 47s
|align="right" | 29h 54m 58s
|align="right" | 213.99Go
|align="right" | 2.0Mo/s
|align="right" | 06m 59s
|align="right" | 01m 05s
|-
|-
| incremental backup: 1.5Go added
| incremental backup: 1.5Go added
Line 133: Line 144:
2014-07-16 01:16:34: burp[5369] Backup completed.
2014-07-16 01:16:34: burp[5369] Backup completed.
2014-07-16 01:16:34: burp[5369] End phase4 (shuffle files)
2014-07-16 01:16:34: burp[5369] End phase4 (shuffle files)
</nowiki></pre>


'''Client 1 full backup 231.99Go'''

<pre><nowiki>
2014-07-17 01:47:37: burp[4654] Client version: 1.3.48
2014-07-17 01:47:37: burp[4654] Begin phase1 (file system scan)
2014-07-17 01:53:24: burp[4654] End phase1 (file system scan)
2014-07-17 01:53:24: burp[4654] Begin phase2 (receive file data)
2014-07-18 07:48:22: burp[4654] End phase2 (receive file data)
2014-07-18 07:48:22: burp[4654] Begin phase3 (merge manifests)
2014-07-18 07:49:27: burp[4654] End phase3 (merge manifests)
2014-07-18 07:49:27: burp[4654] Backup ending - disconnect from client.
2014-07-18 07:49:27: burp[4654] Begin phase4 (shuffle files)
2014-07-18 07:49:27: burp[4654] Doing the atomic data jiggle...
--------------------------------------------------------------------------------
Start time: 2014-07-17 01:47:36
End time: 2014-07-18 07:55:21
Time taken: 01:06:07:45
New Changed Unchanged Deleted Total | Scanned
------------------------------------------------------------
Files: 998898 0 0 0 998898 | 998898
Directories: 74226 0 0 0 74226 | 74226
Soft links: 4531 0 0 0 4531 | 4531
Hard links: 16 0 0 0 16 | 16
Grand total: 1077671 0 0 0 1077671 | 1077671
------------------------------------------------------------

Warnings: 0

Bytes estimated: 228149960334 (212.48 GB)
Bytes in backup: 229772435086 (213.99 GB)
Bytes received: 229772435086 (213.99 GB)
Bytes sent: 0
--------------------------------------------------------------------------------
2014-07-18 07:55:21: burp[4654] Backup completed.
2014-07-18 07:55:21: burp[4654] End phase4 (shuffle files)
</nowiki></pre>
</nowiki></pre>


Revision as of 11:43, 19 July 2014

Burp tests

Description

Version: 1.3.48 (debian package 1.3.48-4~bpo70+1)

Hardware configuration:

Type CPU Disk RAM
Server Marvell Armada 370 (ReadyNAS 104) 2*WDC WD20EFRX-68EUZN0 (soft raid1 using mdadm) 512Mo
Client1 VIA Nano L2007 1600MHz 2*WDC WD20EFRX-68EUZN0 (soft raid1 using mdadm) 4Go
Client2 VIA Nano L2007 1600MHz 2*WDC WD20EFRX-68EUZN0 (soft raid1 using mdadm) 4Go

All hosts have gigabit NICs with properly cabled CAT 6 cables and all are connected to a Gigabit switch (Netgear GS108E).

Ethernet controllers and modules:

  • client: `Realtek Semiconductor Co., Ltd. RTL8111/8168B PCI Express Gigabit Ethernet controller (rev 06)` (controller), `r8168` (module)
  • server: `Marvell 88E1318 Gigabit Ethernet PHY` (controller), `pegasus` (module)

Copying 212Go from client to backup server took 8h 1m 48s (7.50Mo/s). `iperf` measures 890 Mbits/s between server and client.

Test using client1

Description Compression (`compression`) <<FootNote(Note: Extensions which are not compressed: bz, bz2, zip, gz, webm, mov, png, jpg, jpeg, rar, m4v, avi, xv, jar, tgz, mp4, mkv, ogg, flv)>> SSL compression (`ssl_compression`) Duration on client phase 1 phase 2 Data send Throughput (phase 2) Duration on server phase 3
full backup: 209.17Go gzip9 (default value) zlib5 (default value) 34h 17m 34s 07m 18s 34h 10m 18s 167.4Go 1.4Mo/s 06m 22s 02m 22s
full backup: 209.17Go None (`compression=gzip0`) zlib5 (default value) 14h 49m 29s 03m 57s 14h 45m 32s 209.17Go 4.0Mo/s 07m 04s 01m 09s
full backup: 212.47Go None (`compression=gzip0`) None (`ssl_compression=zlib0`) 24h 19m 53s 04m 36s 24h 15m 17s 212.47Go 2.5Mo/s 06m 57s 54s
full backup: 213.99Go None (`compression=gzip0`) None (`ssl_compression=zlib0`) 30h 00m 45s 05m 47s 29h 54m 58s 213.99Go 2.0Mo/s 06m 59s 01m 05s
incremental backup: 1.5Go added None (`compression=gzip0`) None (`ssl_compression=zlib0`) 11m 30s 04m 36s 06m 54s 1.5Go 3.7Mo/s 16m 28s 01m 01s

Logs

Client 1 full backup 212.47Go 

2014-07-14 23:49:34: burp[5369] Client version: 1.3.48
2014-07-14 23:49:34: burp[5369] Begin phase1 (file system scan)
2014-07-14 23:54:10: burp[5369] End phase1 (file system scan)
2014-07-14 23:54:10: burp[5369] Begin phase2 (receive file data)
2014-07-16 01:09:27: burp[5369] End phase2 (receive file data)
2014-07-16 01:09:27: burp[5369] Begin phase3 (merge manifests)
2014-07-16 01:10:31: burp[5369] End phase3 (merge manifests)
2014-07-16 01:10:31: burp[5369] Backup ending - disconnect from client.
2014-07-16 01:10:31: burp[5369] Begin phase4 (shuffle files)
2014-07-16 01:10:31: burp[5369] Doing the atomic data jiggle...
--------------------------------------------------------------------------------
Start time: 2014-07-14 23:49:26
  End time: 2014-07-16 01:16:34
Time taken: 01:01:27:08
                         New   Changed Unchanged   Deleted     Total |  Scanned
                   ------------------------------------------------------------
            Files:    998898         0         0         0    998898 |   998898
      Directories:     74226         0         0         0     74226 |    74226
       Soft links:      4531         0         0         0      4531 |     4531
       Hard links:        16         0         0         0        16 |       16
      Grand total:   1077671         0         0         0   1077671 |  1077671
                   ------------------------------------------------------------

             Warnings:             0

      Bytes estimated:   228124139118 (212.46 GB)
      Bytes in backup:   228135345774 (212.47 GB)
       Bytes received:   228135345774 (212.47 GB)
           Bytes sent:             0
--------------------------------------------------------------------------------
2014-07-16 01:16:34: burp[5369] Backup completed.
2014-07-16 01:16:34: burp[5369] End phase4 (shuffle files)


Client 1 full backup 231.99Go 

2014-07-17 01:47:37: burp[4654] Client version: 1.3.48
2014-07-17 01:47:37: burp[4654] Begin phase1 (file system scan)
2014-07-17 01:53:24: burp[4654] End phase1 (file system scan)
2014-07-17 01:53:24: burp[4654] Begin phase2 (receive file data)
2014-07-18 07:48:22: burp[4654] End phase2 (receive file data)
2014-07-18 07:48:22: burp[4654] Begin phase3 (merge manifests)
2014-07-18 07:49:27: burp[4654] End phase3 (merge manifests)
2014-07-18 07:49:27: burp[4654] Backup ending - disconnect from client.
2014-07-18 07:49:27: burp[4654] Begin phase4 (shuffle files)
2014-07-18 07:49:27: burp[4654] Doing the atomic data jiggle...
--------------------------------------------------------------------------------
Start time: 2014-07-17 01:47:36
  End time: 2014-07-18 07:55:21
Time taken: 01:06:07:45
                         New   Changed Unchanged   Deleted     Total |  Scanned
                   ------------------------------------------------------------
            Files:    998898         0         0         0    998898 |   998898
      Directories:     74226         0         0         0     74226 |    74226
       Soft links:      4531         0         0         0      4531 |     4531
       Hard links:        16         0         0         0        16 |       16
      Grand total:   1077671         0         0         0   1077671 |  1077671
                   ------------------------------------------------------------

             Warnings:             0

      Bytes estimated:   228149960334 (212.48 GB)
      Bytes in backup:   229772435086 (213.99 GB)
       Bytes received:   229772435086 (213.99 GB)
           Bytes sent:             0
--------------------------------------------------------------------------------
2014-07-18 07:55:21: burp[4654] Backup completed.
2014-07-18 07:55:21: burp[4654] End phase4 (shuffle files)


Old tests

There was a network configuration problem: `r8169` module was used instead of `r8168` on the client.

Test using client1

Description Compression SSL compression Duration on client (phase 1/phase 2) Throughput (phase 2)
First backup: 210Go None (`compression=gzip0`) (default parameter `ssl_compression` not set) 37h 09m 30s (18m 56s/36h 50m 34s) 1.6 Mo/s
Second backup: 34.33Mo added None (`compression=gzip0`) (default parameter `ssl_compression` not set) 00h 18m 10s (16m 45s/1m 25s) 0.4 Mo/s
Third backup: 4.14Go (mp4 files) added None (`compression=gzip0`) (default parameter `ssl_compression` not set) 01h 19m 54s (11m 56s/1h 07m 58s) 1.03 Mo/s
Fourth backup: 4.24Go (mp4 files) added None (`compression=gzip0`) `ssl_compression=gzip0` 00h 58m 16s (16m 39s/41m 37s) 1.73 Mo/s

Test using client2

Description Compression SSL compression Duration on client (phase 1/phase 2) Throughput (phase 2)
First backup: None (`compression=gzip0`) (default parameter `ssl_compression` not set) 37h 06m 53s (15m 40s/36h 51m 13s)
Retrieved from "https://wiki.duckcorp.org//index.php?title=Burp&oldid=209"