Burp
From DcSharedWiki
Burp tests
Description
Version: 1.3.48 (debian package 1.3.48-4~bpo70+1)
Hardware configuration:
| Type | CPU | Disk | RAM |
| Server | Marvell Armada 370 (ReadyNAS 104) | 2*WDC WD20EFRX-68EUZN0 (soft raid1 using mdadm) | 512Mo |
| Client1 | VIA Nano L2007 1600MHz | 2*WDC WD20EFRX-68EUZN0 (soft raid1 using mdadm) | 4Go |
| Client2 | VIA Nano L2007 1600MHz | 2*WDC WD20EFRX-68EUZN0 (soft raid1 using mdadm) | 4Go |
All hosts have gigabit NICs with properly cabled CAT 6 cables and all are connected to a Gigabit switch (Netgear GS108E).
Ethernet controllers and modules:
- client: `Realtek Semiconductor Co., Ltd. RTL8111/8168B PCI Express Gigabit Ethernet controller (rev 06)` (controller), `r8168` (module)
- server: `Marvell 88E1318 Gigabit Ethernet PHY` (controller), `pegasus` (module)
Context:
- Copying 212Go from client to backup server using scp took 8h 1m 48s (7.50Mo/s).
- `iperf` measures 890 Mbits/s between server and client.
- Copying 244.31Go grom client to backup server using rsync took 36h 21m (sent 189.41G bytes - received 25.48M bytes - 1.45M bytes/sec).
Test using client1
| Description | Compression (`compression`) <<FootNote(Note: Extensions which are not compressed: bz, bz2, zip, gz, webm, mov, png, jpg, jpeg, rar, m4v, avi, xv, jar, tgz, mp4, mkv, ogg, flv)>> | SSL compression (`ssl_compression`) | Duration on client | phase 1 | phase 2 | Data send | Throughput (phase 2) | Duration on server | phase 3 |
| full backup: 209.17Go | gzip9 (default value) | zlib5 (default value) | 34h 17m 34s | 07m 18s | 34h 10m 18s | 167.4Go | 1.4Mo/s | 06m 22s | 02m 22s |
| full backup: 209.17Go | None (`compression=gzip0`) | zlib5 (default value) | 14h 49m 29s | 03m 57s | 14h 45m 32s | 209.17Go | 4.0Mo/s | 07m 04s | 01m 09s |
| full backup: 214.27Go | None (`compression=gzip0`) | zlib5 (default value) | 30h 05m 44s | 7m 45s | 29h 57m 59s | 214.27Go | 2.0Mo/s | 06m 56s | 01m 01s |
| full backup: 212.47Go | None (`compression=gzip0`) | None (`ssl_compression=zlib0`) | 24h 19m 53s | 04m 36s | 24h 15m 17s | 212.47Go | 2.5Mo/s | 06m 57s | 54s |
| full backup: 213.99Go | None (`compression=gzip0`) | None (`ssl_compression=zlib0`) | 30h 00m 45s | 05m 47s | 29h 54m 58s | 213.99Go | 2.0Mo/s | 06m 59s | 01m 05s |
| incremental backup: 1.5Go added | None (`compression=gzip0`) | None (`ssl_compression=zlib0`) | 11m 30s | 04m 36s | 06m 54s | 1.5Go | 3.7Mo/s | 16m 28s | 01m 01s |
Logs
Client 1 full backup 212.47Go
2014-07-14 23:49:34: burp[5369] Client version: 1.3.48
2014-07-14 23:49:34: burp[5369] Begin phase1 (file system scan)
2014-07-14 23:54:10: burp[5369] End phase1 (file system scan)
2014-07-14 23:54:10: burp[5369] Begin phase2 (receive file data)
2014-07-16 01:09:27: burp[5369] End phase2 (receive file data)
2014-07-16 01:09:27: burp[5369] Begin phase3 (merge manifests)
2014-07-16 01:10:31: burp[5369] End phase3 (merge manifests)
2014-07-16 01:10:31: burp[5369] Backup ending - disconnect from client.
2014-07-16 01:10:31: burp[5369] Begin phase4 (shuffle files)
2014-07-16 01:10:31: burp[5369] Doing the atomic data jiggle...
--------------------------------------------------------------------------------
Start time: 2014-07-14 23:49:26
End time: 2014-07-16 01:16:34
Time taken: 01:01:27:08
New Changed Unchanged Deleted Total | Scanned
------------------------------------------------------------
Files: 998898 0 0 0 998898 | 998898
Directories: 74226 0 0 0 74226 | 74226
Soft links: 4531 0 0 0 4531 | 4531
Hard links: 16 0 0 0 16 | 16
Grand total: 1077671 0 0 0 1077671 | 1077671
------------------------------------------------------------
Warnings: 0
Bytes estimated: 228124139118 (212.46 GB)
Bytes in backup: 228135345774 (212.47 GB)
Bytes received: 228135345774 (212.47 GB)
Bytes sent: 0
--------------------------------------------------------------------------------
2014-07-16 01:16:34: burp[5369] Backup completed.
2014-07-16 01:16:34: burp[5369] End phase4 (shuffle files)
Client 1 full backup 231.99Go
2014-07-17 01:47:37: burp[4654] Client version: 1.3.48
2014-07-17 01:47:37: burp[4654] Begin phase1 (file system scan)
2014-07-17 01:53:24: burp[4654] End phase1 (file system scan)
2014-07-17 01:53:24: burp[4654] Begin phase2 (receive file data)
2014-07-18 07:48:22: burp[4654] End phase2 (receive file data)
2014-07-18 07:48:22: burp[4654] Begin phase3 (merge manifests)
2014-07-18 07:49:27: burp[4654] End phase3 (merge manifests)
2014-07-18 07:49:27: burp[4654] Backup ending - disconnect from client.
2014-07-18 07:49:27: burp[4654] Begin phase4 (shuffle files)
2014-07-18 07:49:27: burp[4654] Doing the atomic data jiggle...
--------------------------------------------------------------------------------
Start time: 2014-07-17 01:47:36
End time: 2014-07-18 07:55:21
Time taken: 01:06:07:45
New Changed Unchanged Deleted Total | Scanned
------------------------------------------------------------
Files: 998898 0 0 0 998898 | 998898
Directories: 74226 0 0 0 74226 | 74226
Soft links: 4531 0 0 0 4531 | 4531
Hard links: 16 0 0 0 16 | 16
Grand total: 1077671 0 0 0 1077671 | 1077671
------------------------------------------------------------
Warnings: 0
Bytes estimated: 228149960334 (212.48 GB)
Bytes in backup: 229772435086 (213.99 GB)
Bytes received: 229772435086 (213.99 GB)
Bytes sent: 0
--------------------------------------------------------------------------------
2014-07-18 07:55:21: burp[4654] Backup completed.
2014-07-18 07:55:21: burp[4654] End phase4 (shuffle files)
Client1 full backup: 214.27Go
2014-07-19 18:32:40: burp[9606] Client version: 1.3.48
2014-07-19 18:32:40: burp[9606] Begin phase1 (file system scan)
2014-07-19 18:40:25: burp[9606] End phase1 (file system scan)
2014-07-19 18:40:25: burp[9606] Begin phase2 (receive file data)
2014-07-21 00:38:24: burp[9606] End phase2 (receive file data)
2014-07-21 00:38:24: burp[9606] Begin phase3 (merge manifests)
2014-07-21 00:39:25: burp[9606] End phase3 (merge manifests)
2014-07-21 00:39:25: burp[9606] Backup ending - disconnect from client.
2014-07-21 00:39:25: burp[9606] Begin phase4 (shuffle files)
2014-07-21 00:39:25: burp[9606] Doing the atomic data jiggle...
--------------------------------------------------------------------------------
Start time: 2014-07-19 18:32:39
End time: 2014-07-21 00:45:20
Time taken: 01:06:12:41
New Changed Unchanged Deleted Total | Scanned
------------------------------------------------------------
Files: 998898 0 0 0 998898 | 998898
Directories: 74226 0 0 0 74226 | 74226
Soft links: 4531 0 0 0 4531 | 4531
Hard links: 16 0 0 0 16 | 16
Grand total: 1077671 0 0 0 1077671 | 1077671
------------------------------------------------------------
Warnings: 0
Bytes estimated: 230074235176 (214.27 GB)
Bytes in backup: 230074235177 (214.27 GB)
Bytes received: 230074235177 (214.27 GB)
Bytes sent: 0
--------------------------------------------------------------------------------
2014-07-21 00:45:20: burp[9606] Backup completed.
2014-07-21 00:45:20: burp[9606] End phase4 (shuffle files)
Old tests
There was a network configuration problem: `r8169` module was used instead of `r8168` on the client.
Test using client1
| Description | Compression | SSL compression | Duration on client (phase 1/phase 2) | Throughput (phase 2) |
| First backup: 210Go | None (`compression=gzip0`) | (default parameter `ssl_compression` not set) | 37h 09m 30s (18m 56s/36h 50m 34s) | 1.6 Mo/s |
| Second backup: 34.33Mo added | None (`compression=gzip0`) | (default parameter `ssl_compression` not set) | 00h 18m 10s (16m 45s/1m 25s) | 0.4 Mo/s |
| Third backup: 4.14Go (mp4 files) added | None (`compression=gzip0`) | (default parameter `ssl_compression` not set) | 01h 19m 54s (11m 56s/1h 07m 58s) | 1.03 Mo/s |
| Fourth backup: 4.24Go (mp4 files) added | None (`compression=gzip0`) | `ssl_compression=gzip0` | 00h 58m 16s (16m 39s/41m 37s) | 1.73 Mo/s |
Test using client2
| Description | Compression | SSL compression | Duration on client (phase 1/phase 2) | Throughput (phase 2) |
| First backup: | None (`compression=gzip0`) | (default parameter `ssl_compression` not set) | 37h 06m 53s (15m 40s/36h 51m 13s) |
