File: Changelog

package info (click to toggle)
virtnbdbackup 2.42-1
  • links: PTS, VCS
  • area: main
  • in suites: forky, sid
  • size: 672 kB
  • sloc: python: 4,426; makefile: 9
file content (1455 lines) | stat: -rw-r--r-- 45,665 bytes parent folder | download
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
1001
1002
1003
1004
1005
1006
1007
1008
1009
1010
1011
1012
1013
1014
1015
1016
1017
1018
1019
1020
1021
1022
1023
1024
1025
1026
1027
1028
1029
1030
1031
1032
1033
1034
1035
1036
1037
1038
1039
1040
1041
1042
1043
1044
1045
1046
1047
1048
1049
1050
1051
1052
1053
1054
1055
1056
1057
1058
1059
1060
1061
1062
1063
1064
1065
1066
1067
1068
1069
1070
1071
1072
1073
1074
1075
1076
1077
1078
1079
1080
1081
1082
1083
1084
1085
1086
1087
1088
1089
1090
1091
1092
1093
1094
1095
1096
1097
1098
1099
1100
1101
1102
1103
1104
1105
1106
1107
1108
1109
1110
1111
1112
1113
1114
1115
1116
1117
1118
1119
1120
1121
1122
1123
1124
1125
1126
1127
1128
1129
1130
1131
1132
1133
1134
1135
1136
1137
1138
1139
1140
1141
1142
1143
1144
1145
1146
1147
1148
1149
1150
1151
1152
1153
1154
1155
1156
1157
1158
1159
1160
1161
1162
1163
1164
1165
1166
1167
1168
1169
1170
1171
1172
1173
1174
1175
1176
1177
1178
1179
1180
1181
1182
1183
1184
1185
1186
1187
1188
1189
1190
1191
1192
1193
1194
1195
1196
1197
1198
1199
1200
1201
1202
1203
1204
1205
1206
1207
1208
1209
1210
1211
1212
1213
1214
1215
1216
1217
1218
1219
1220
1221
1222
1223
1224
1225
1226
1227
1228
1229
1230
1231
1232
1233
1234
1235
1236
1237
1238
1239
1240
1241
1242
1243
1244
1245
1246
1247
1248
1249
1250
1251
1252
1253
1254
1255
1256
1257
1258
1259
1260
1261
1262
1263
1264
1265
1266
1267
1268
1269
1270
1271
1272
1273
1274
1275
1276
1277
1278
1279
1280
1281
1282
1283
1284
1285
1286
1287
1288
1289
1290
1291
1292
1293
1294
1295
1296
1297
1298
1299
1300
1301
1302
1303
1304
1305
1306
1307
1308
1309
1310
1311
1312
1313
1314
1315
1316
1317
1318
1319
1320
1321
1322
1323
1324
1325
1326
1327
1328
1329
1330
1331
1332
1333
1334
1335
1336
1337
1338
1339
1340
1341
1342
1343
1344
1345
1346
1347
1348
1349
1350
1351
1352
1353
1354
1355
1356
1357
1358
1359
1360
1361
1362
1363
1364
1365
1366
1367
1368
1369
1370
1371
1372
1373
1374
1375
1376
1377
1378
1379
1380
1381
1382
1383
1384
1385
1386
1387
1388
1389
1390
1391
1392
1393
1394
1395
1396
1397
1398
1399
1400
1401
1402
1403
1404
1405
1406
1407
1408
1409
1410
1411
1412
1413
1414
1415
1416
1417
1418
1419
1420
1421
1422
1423
1424
1425
1426
1427
1428
1429
1430
1431
1432
1433
1434
1435
1436
1437
1438
1439
1440
1441
1442
1443
1444
1445
1446
1447
1448
1449
1450
1451
1452
1453
1454
1455
Version 2.42
---------
 * virtnbdrestore: add --compress option: if enabled, NBD server used for
 restore will make use of the qcow compression driver.  This way, restored
 data within the qcow image is compressed according to the image compression
 type setting.

 This will be beneficial for images that contained many compressed blocks, as
 the restored images will be way smaller. (#295)

Version 2.41
---------
 * Disable building rust bindings when creating venv (#293)
 * Small code cleanups
 * Use tqdm logging_redirect_tqdm to not break progress bars (#294) 

Version 2.40
---------
 * Add OpenSUSE 16 to build scripts
 * Use trixie for docker container
 * venv/create.sh fails to successfully build libnbd without python setuptools
 (#290)
 * Add support for VIR_DOMAIN_BACKUP_BEGIN_PRESERVE_SHUTDOWN_DOMAIN: As with
 libvirt version 11.10, this flag prevents VM shutdown while an backup job is
 still active. The VM is in that scenario reset and paused instead of
 terminated allowing the backup to finish. Once the backup finishes the VM
 process is terminated. See:

  https://libvirt.org/kbase/live_full_disk_backup.html#shutdown-of-the-guest-os-during-backup

Version 2.39
---------
 * Add fedora 42 to build scripts.
 * Gracefully handle network attached disk devices, show error message and
 hint (#286)

Version 2.38
---------
 [Francisco Javier]
 * virtnbdbackup: reuse existing sftp connections during backup. (#283)

 [Michael Ablassmeier]
 * virtnbdmap: replaying changes may fail with "[Errno 22] Invalid argument":
 the NBD device sometimes needs some time to settle, wait until it is seekable
 to the end of the block list instead of failing.
 * virtnbdbackup: During full backup, all old checkpoints are removed.
 Sometimes there might be situations where bitmaps are not existent anymore.
 Now, during full backup, checkpoint consistency is validated prior to
 removal, and checkpoints tied to  non-existent bitmaps will be removed with
 VIR_DOMAIN_CHECKPOINT_DELETE_METADATA_ONLY flags enabled, to allow flawless
 full backups. The utility will issue WARNINGS in this case.

 See discussion (#285)

Version 2.37
---------
 * Add missing nbdkit and python plugin to docker container image. (#282)

Version 2.36
---------
 * virtnbdbackup: add `--pause` option: Suspend and resume virtual machine
 while starting backup job. This can be useful for virtual machines where
 freezing file systems using the qemu guest agent is not possible.

Version 2.35
---------
 * virtnbdrestore: fix restore issue in case virtual machine is using
 volume based disk configuration: the restore process attempted to identify
 the volumes which do not exist anymore. Adjust virtual machine config and
 reset the volume base setting to a file based notation. (#280)

Version 2.34
---------
 * virtnbdbackup: show hint about virtual machine config adjustment
 possibility in case raw devices are attached.
 * virtnbdrestore: do not show warning if no data-file setting is found in
 qcow json file.
 * Add RHEL10 RPM build to github workflow
 * Do not make vm6.tests fail if /tmp/datafile.raw is missing.

Version 2.33
---------
 * virtnbdrestore: Add support for data-file setting in qcow images:

 As with versions (>= v10.10.0), libvirt supports the qcow image data-file
 setting. This allows to create metadata QCOW Images for RAW disk images or
 direct attached LVM/ZFS/RBD volumes.

 Bitmaps are then stored within the metadata QCOW image, allowing to create
 incremental backups for these disks, too.

 During restore, the QCOW image must be created with the correct data-file
 setting and the virtual machine configuration must be adopted. The utility
 will now adjust paths accordingly if -c option is specified.

 See README for details on how to configure your virtual machine to support
 incremental backups with RAW devices.

Version 2.32
---------
 * virtnbdrestore: handle exception if files in the backup folder are missing,
 show warning instead (#277)

Version 2.31
---------
 * virtnbdrestore: remove raw devices from xml config only if option `--raw`
 is not specified (#276)

Version 2.30
---------
 * virtnbdbackup: handle lost CBT case (#272): fallback to full backup in case
 stored dirty bitmaps in the qcow images are not sync with libvirt
 checkpoints. (Denis V. Lunev <den@openvz.org>)

Version 2.29
---------
 * virtnbdrestore: wrong vm config XML used if until option is used (#270)

Version 2.28
---------
 * virtnbdbackup:
 [Denis V. Lunev <den@openvz.org>]
 - improve overlap() performance. The call should be O(L) complexity (#265)

Version 2.27
---------
 * virtnbdbackup:
 - Skip saving checksum information for raw images included with --raw option:
 The checksum calculated does not actually match during verify because certain
 parts of the images may be seeked instead of actually written. (#269)

Version 2.26
---------
 * virtnbdbackup:
 - Add info log message before and after extent query has finished.
 - Enhance function to detect sparse/fstrimmed regions (#263).
 - Enhance CI tests

Version 2.25
---------
 * virtnbdbackup:
  - Detect QCOW2 formatted direct attached RAW Volumes (#264):

  Some solutions such as OpenNebula will use direct attached LVM volumes in
  virtual machines. These LVM volumes were treated like a RAW device, but
  allow all required operations for full/incremental backup.

  Changes introduced in v2.18 would exclude these devices from backup even if
  `--raw` option is passed. Now during disk detection, use the disk format in
  the driver type setting to correctly detect a lvm volume backed by qcow2,
  and handle them just like regular qcow based images.

Version 2.24
---------
 * virnbdbackup:
  - Add option --no-sparse-detection (#263): The feature implemented as
  with version 2.21 requires to query the complete base:allocation
  bitmap during incremental backup.

  Depending on the Hypervisor hardware and the virtual machine disk size,
  querying the base:allocation bitmap may take longer than to simply treat
  fstrimmed blocks the same way as changed blocks and include them in the
  backup.

  The option introduced can be used to disable the new behavior and can
  enhance backup times for specific users.

Version 2.23
---------
 * virtnbdbackup:
  - Log created nbd socket information in URI format, so it can be reused
  with external tools more easily.
  - Add human readable backup sizes in log output.

Version 2.22
---------
 * Test enhancements:
   - check btrfs filesystem stats in restored VM.
   - run checksum verify after all backups.
 * virtnbdbackup:
   - If NBD connection drops, catch correct exception. (#254)
   - Provide better error message if checkpoint inconsistency is detected.

Version 2.21
---------
 * virtnbdbackup: Detect dirty regions containing only zeroes during
 incremental backup (fstrim) (#250)

 Before, any dirty block being part of the incremental/differential bitmap
 would be backed up, even if not required. Thanks to the input and
 enhancements
 of
	Roman Grigoriev <grigoriev177 _at_ gmail.com>
 and
	Denis V. Lunev <den _at_ openvz.org>
 the zeroed block regions are now compared to the base:allocation
 bitmap and detected as such.

 This reduces the amount of data backed up if virtual machines have disks
 configured with discard=unmap option and use fstrim to free unused space.

Version 2.20
---------
 * virtnbdmap: add option -H for better debugging, code cleanup
 * virtnbdmap: fix situations where multiple blocks in row are
 required to map the correct data.

Version 2.19
---------
 * virtnbdmap sometimes fails to correctly map disk images due to
 block offsets spanning multiple frames in the stream format, resulting in
 Input/Output errors accessing the disk partition data (#249)

Version 2.18
---------
 * If auto backup mode is switched to incremental and --raw option
 is passed, backup would fails instead of showing an warning that
 raw devices are excluded (#239)

Version 2.17
---------
 * Add rpm package build for fedora 41 (#234)
 * virtnbdbackup: Remote backup fails if spaces exist in image path (#235)
 * virtnbdrestore: Remote restore would fail if target path contains spaces.

Version 2.16
---------
 * virtnbdrestore: If domain with multiple disks is restored and config
 adjustment is requested, only the path to the last disk is updated. (#232).
 Thanks Richard Stephens for report and fix.
 * virtnbdrestore: fix logic for adjusting the vm name

Version 2.15
---------
 * Update Dockerfile and README
 * virtnbdrestore: use absolute path to disk files during local restore.
 * virtnbdrestore: make absolute path mandatory for remote restore.

Version 2.14
---------
 * virtnbdrestore: Add option -A (--preallocate) to create thick, full
 pre-allocated target images during restore.

Version 2.13
---------
 * virtnbdbackup: If libvirt daemon is unreachable, it would fail with an
 misleading error code due exception not raised.
 * virtnbdbackup: setup callback function using registerCloseCallback, if
 libvirt connection drops during backup, re-establish the connection in order
 to stop leftover backup tasks
 * virtnbdbackup: catch exceptions if nbd read fails.
 * virtnbdrestore: add option -C to adjust config file name during restore.

Version 2.12
---------
 * Update README
 * virtnbdrestore: fix exception if no $HOME environment variable is set
 (#197)

Version 2.11
---------
 * Update README
 * Raise NBD connection error if `--tls` option is set but installed libnbd
 bindings for python do not support required features.
 * Set TLS connection options to TLS_REQUIRE instead of TLS_ALLOW which would
 fallback to non-encrypted data transfer.
 * If LIBNBD_DEBUG is set, add NBD trace messages to logfile.

Version 2.10
---------
 * Add --ssh-port option.

Version 2.9
---------
 * Fix: backup with compression enabled fails: unsupported operand type(s) for
 +: 'int' and 'dict' (#177)

Version 2.8
---------
 * Add packages compatible to fedora 39 to package build (#174)
 * Show total saved disk size in human readable output (#173)

Version 2.7
---------
 * Update README
 * End backup with warning if software emulated TPM device is attached
 (#169)
 * Detect remote connection based on libvirt URI, checking hostnames could lead
 to situation where local backup is detected as remote backup (#170)

Version 2.6
---------
 * Fix IndexError exception if auth file is used in qemu uri (#167)
 * Credential function for libvirt must return integer: fix NoneType
 exception if actual libvirt authentication is required.
 * Simplify libvirt authentication code: attempt to use SASL based mechanism
 only if --user and --password options are set.
 * If authentication fails because of missing SASL mechs, ajdust error
 message, provide hint for --user and --password options.
 * Update README regards OVIRT/RHEV/OLVM (no mechanism available: No worthy
 mechs found)

Version 2.5
---------
 * Move some log messages from info to debug loglevel
 * Log information about libnbd version only once
 * Catch command not found error during remote backup if qemu-img
 is missing: change loglevel to warning.
 * Catch command not found error during remote restore: fail with
 proper error message.
 * If no qcow image info has been created during backup, issue warning
 during restore that default options are used.
 * Update README: add note about scratch files.
 * Do not attempt to freeze filesystems if virtual machine is in paused state
 (#166)

Version 2.4
---------
 * Update RADME
 * Add qemu-utils to package dependencies: if installed on a system without
 libvirt/qemu, backup fails because of missing qemu-img executable.
 * Add openssh-client to package dependencies: required for remote libvirtd
 connection.
 * Catch exception if executed commands such as qemu-img are missing on
 system.
 * Code cleanup

Version 2.3
---------
 * Update README
 * Add option -S (--start-domain): if specified and virtual domain is offline
 during backup, domain will be started in pause mode, allowing to execute
 full/diff/inc backups. Domain is destroyed as soon as operation finished
 by using libvirt's AUTODESTROY flag. (#164)
 * Move code for preflight tests to separate module.

Version 2.2
---------
 * Fix Progressbar during restore: wrong values used. (#160)
 * Catch exception if during restore connection to NBD server fails (#163)
 * Provide better info message what NBD connection is waiting for.
 * Call flush() on NBD connection during restore: restore of domains
 with multiple disks could fail with NbdConnectionTimeout due to race
 condition (#163)
 * Pass pidFile to qemu-nbd process for local NBD server during restore,
 report PID of forked process instead of parent.

Version 2.1
---------
 * Fix Progressbar: Since change for issue #133 the Progressbar was
 updated with wrong values and as result progressed too fast for the amount of
 data actually written. (#160)

Version 2.0
---------
 * Update README
 * Fail early during incremental or differential backup if no
 full backup is found in target directory.

Version 1.9.55
---------
 * virtnbdmap: pass listen address argument to qemu-nbd command.

Version 1.9.54
---------
 * Exit with error if both compress option and raw output format
 is specified: options are mutually exclusive.
 * Update progressbar during chunked data read/write. If big portions
 of the disk are backed up, it would appear as if no progress is made
 (#133)
 * If an active remote backup is aborted (ctrl+c), stopping the libvirt backup job
 failed because the signal is sent to the used ssh connection, too. Now the
 signal handler attempts to re establish the connection to the remote system
 in order to correctly stop the backup operation.
 * Add close() function to libvirt client class: disconnect libvirt after
 backup has finished.
 * virtnbdmap: code reorg.

Version 1.9.53
---------
 * Move checksum related code to own function
 * Set a shorter thread name, makes logging messages are easier to read.
 * Add opensuse/leap:15.5 to release build.

Version 1.9.52
---------
 * Code cleanup
 * Build release packages using github actions.

Version 1.9.51
---------
 * Slight change in log message wording.
 * virtnbdrestore: if starting NBD service on remote system fails, catch
 exceptions accordingly.
 * virtnbdrestore: if --nbd-ip parameter is set, pass -b option to qemu-nbd
 command so the NDB service binds to specified IP address.
 * virtnbdrestore: Set SSH session mode to UPLOAD, otherwise uploading
 files such as UEFI images and vm config fails.
 * virtnbdrestore: Catch exception if connection to libvirt fails
 * virtnbdrestore: Create target directory on remote system only if it does
 not exist, Thanks @draggeta
 * CI: enhance remote backup check: test if UEFI loader files are copied
 correctly during remote backup.

Version 1.9.50
---------
 * Add openssh-client to docker image required for remote backup
 functionality (#151)
 * Check if specified ip address in `--nbd-ip` parameter is an
 ipv6 address and if so, use ipv6 ip notation for NBD client
 connection (#150)
 * Fix NBD connection timeout: current implementation only waited
 until NBD server on socket was reachable. Now it also attempts to
 retry connection for remote TCP NBD servers (#150)
 * Fix exception in ssh client during raise

Version 1.9.49
---------
 * Logging: Use blue color definition instead of light-blue which might not be
 supported by all colorlog versions (#148)

Version 1.9.48
---------
 * Rework some error messages / exception handling.

Version 1.9.47
---------
 * Remote backup: catch more exceptions accordingly, provide better
 error message if ssh connection fails.

Version 1.9.46
---------
 * Update package dependencies in docker file.
 * Spelling fixes

Version 1.9.45
---------
 * virtnbdbackup: Add more info output during checkpoint handling, move
 error message.
 * Add colorful log messages so warnings and and errors are easier to spot:
 can be disabled using the --nocolor option.

Version 1.9.44
---------
 * virtnbdbackup: disable progressbar if quiet option is enabled.

Version 1.9.43
---------
 * virtnbdbackup: add --quiet option: disable log output to stderr,
 log into logfile only. (#137)

Version 1.9.42
---------
 * virtnbdbackup: skip checksum output if backup is redirected to stdout

Version 1.9.41
---------
 * virtnbdrestore: add Option -B to allow changing default buffer size
 during verify to speed up process.

Version 1.9.40
---------
 * Update README
 * New feature: compute adler32 checksum for written data and add
 verify option to virtnbdrestore, so its possible to check for corrupt
 backups without having to restore.

Version 1.9.39
---------
 * Update README

Version 1.9.38
---------
 * Release packages compatible with RHEL9 (#130)
 + add required vagrant file
 + adjust requirements during package build
 + put dist files into separate directories

Version 1.9.37
---------
 * Update README: add example how to convert qcow images to the required
 format to support all backup features.

Version 1.9.36
---------
 * Fail with better understandable error message if libvirt version requires
 the virtual machine configuration to be adjusted for backup to work.

Version 1.9.35
---------
 * Fix some pylint warnings, add ignores.

Version 1.9.34
---------
 * Update README

Version 1.9.33
---------
 * Update README

Version 1.9.32
---------
 * Fix TypeError during backup of transient virtual machines: do not pass
 flags to checkpointCreateXML as list. (#122)
 * Add simple test for transient virtual machine handling.

Version 1.9.31
---------
 * setup.cfg: fix obsolete description-file warning
 * In case domain has an already active block job, add note about
 option -k to the error message (#120).

Version 1.9.30
---------
 * Update README

Version 1.9.29
---------
 * Update README

Version 1.9.28
---------
 * Remote backup: enhance logging if executing remote command via SSH fails.

Version 1.9.27
---------
 * Enhance debug logging for remote backup.
 * Update README regarding remote backup and its requirements. (#117)

Version 1.9.26
---------
 * virtnbdbackup: transient vm backup: validate the state of the checkpoint
   during redefine: this can help to detect situations where the bitmap is
   broken.
 * virtnbdmap: Fix typo in log message.
 * Update README regarding port usage during remote backups.
 * Add notes regarding stability of the features, thanks @pipo

Version 1.9.25
---------
 * Fix typo in log messages.

Version 1.9.24
---------
 * Remove leftover pid files in /var/tmp during offline backup (#114)

Version 1.9.23
---------
 * Code cleanups
 * Spelling fixes, run codespell during CI

Version 1.9.22
---------
 * Code cleanups, pylint warning
 * Update copyright stanca

Version 1.9.21
---------
 * Code cleanups

Version 1.9.20
---------
 * More code cleanup and reorg
 * Module directory is now pylint clean (except for some ignores): run pylint
  using github actions.

Version 1.9.19
---------
 * Code cleanup: move some functions to separate files.

Version 1.9.18
---------
 * virtnbdbackup: treat direct attached lun and block devices just like
 regular raw disks: backup will work but only in backup mode copy and
 of course full provisioned backup of these block devices is created
 if option `--raw` is enabled. (#111)

Version 1.9.17
---------
 * virtnbdrestore: fix restore fails with IndexError if CDROM or LUN devices
 appear in the list of devices before the first disk. (#110).

Version 1.9.16
---------
 * virtnbdrestore: use disk size from the latest backup during restore: handle
 situations where disk size has changed between full and incremental or
 differential backups (#109)

Version 1.9.15
---------
 * Add install_requires to setup.py
 * Backup and restore autostart setting for domain if set. (#106)

Version 1.9.14
---------
 * Adjust log messages during disk parsing.
 * Ignore direct attached block devices which use disk type notation (#98)

Version 1.9.13
---------
 * Fallback to backup mode copy if virtual machine has only raw disks
 attached (#94)
 * Some fixes in regard on how attached 'raw' disks are handled:
  + fail correctly if backup to stdout is attempted
  + override --raw option during attempted incremental or differential backup:
  backup modes will only be attempted for disks supporting the required
  features.
 * If no disks suitable for backup are detected or backup fails, save at least
 virtual machine configuration and related kernel/uefi files (#93)

Version 1.9.12
---------
 * Rework logging facilities: use local log instance of global logging object
 to have saner logging information.

Version 1.9.11
---------
 * Log traceback for unknown exceptions for easier analysis of issues. (#92)

Version 1.9.10
---------
 * Remote incremental backup fails: checkpoint virtnbdbackup.1 already exists:
 check for checkpoint file only on local system, not on the remote system
 (#89)

Version 1.9.9
---------
 * Fix exception handling/raise in case bitmap already exists. (#85)

Version 1.9.8
---------
 * Code cleanup

Version 1.9.7
---------
 * Update README
 * Code cleanup, use shorter class names, change some error handling.
 * virtnbdrestore: in case option -D is passed, set option to adjust virtual
 machine config automatically.
 * virtnbdbackup: amount of ports used for offline backup depends on how many
 workers are used, not how much disks are attached, log correct port range use
 during offline backup.

Development:
 * Github workflow now saves all files created by testsuite for better
 analysis, extend some testcases.


Version 1.9.6
---------
 * virtnbdrestore: add detection for disks with volume notation, reset
 type to file during restore so virtual machine config is adjusted to
 new restore paths. (#81)


Version 1.9.5
---------
 * virtnbdrestore and virtnbdmap: code cleanups, add type annotations.


Version 1.9.4
---------
 * Code cleanup: more type annotations, correctly catch ssh exceptions.
 * Better logging information in case of remote backup
 * Remote backup of offline virtual machines with multiple disks may fail:
 (#82) port used by remote NBD Service must be unique.  Also report used
 ports in logfile.


Version 1.9.3
---------
 * Move function call for backing up qemu disk information outside worker
 threads: should fix occasional "Fatal Python error: _enter_buffered_busy"
 error during backup.


Version 1.9.2
---------
 * Return function if gathering qemu disk info fails for some reason.
 * If remote backup is saved to local zip file, backup may fail because
 absolute path to additional loader/nvram files is wrong.
 * Fix naming if copy backup is saved to zip file output: dont append
 checkpoint information to created files.


Version 1.9.1
---------
 * Code cleanup: fix pylint warning, remove obsolete p.wait() calls with
 hardcoded time limit, start adding type annotations, reworked imports.
 * Fix disk detection with volume notation. (#81)
 * Backup and restore qcow image related settings: during backup
 qemu-img info is called on qcow based image devices and qcow specific options
 are stored as json dump. If information is existent, on restore the
 original settings of the qcow image (such as cluster size or other options)
 are now applied during image creation. (#80)
 * Fix some race conditions during restore of multiple disks and sequences: NBD
 connection handle was not closed, following qemu-nbd calls would fail due
 to locked files or already in use tcp ports.


Development:
 * Change github workflows: now use ubuntu 22.04 and add test for remote
 backup via localhost.


Version 1.9
---------
 * Update README
 * Make --compress option configurable, it is now possible to set
 compression level (--compress=X) (#77)
 * Fix offline backup of virtual machines with multiple disks attached: nbd
 connection would use wrong socket file. (#76)
 * Fix offline backup of remote vms: set missing socket file parameter.


Version 1.8.2
---------
 * Code cleanup
 * Fix Issue (#74): Remote backup fails to copy [loader] file

Version 1.8.1
---------
 * Update manpages to be more debian compliant
 * Add manpages to manifest file, too

Version 1.8
---------
 * Add manpages

Version 1.7
---------
 * Code cleanup

Version 1.6
---------
* virtnbdbackup:
	+ add --syslog option: enable log output to system syslog
	facility.

Version 1.5
---------
* virtnbdrestore:
	+ add --logfile option (defaults to $HOME)
	+ print actual json if dumping saveset contents
* virtnbdmap:
	+ add --logfile option (defaults to $HOME)

Version 1.4
---------
* If backup/restore is executed as regular user, set default uri to
qemu:///session.

Version 1.3
---------
* Code cleanup
* Update README

Version 1.2
---------
* Code cleanup

Version 1.1
---------
* Code cleanup
* Add warning if reading checkpoint information with size fails.

Version 1.0
---------
* Code cleanup
* (#69) Rework --printonly option: use checkpoint size as reported by libvirt
for estimating next incremental or differential backup size.
* (#70) Add --threshold option for incremental / differential backup 

Version 0.99
---------
* Add python3-paramiko to debian build dependencies (#68)

Version 0.98
---------
* Add missing license/copyright headers

Version 0.97
---------
* Update README

Version 0.96
---------
* Code cleanup
* Add MANIFEST.in: add additional files like dockerfile to source dist
package.
* Add LICENSE and Changelog file to rpm/debian package distribution configs.

Version 0.95
---------
* virtnbdrestore: code cleanup
* Add updated dockerfile based on work by Adrián Parilli
<a.parilli@staffwerke.de> to the repository.
* Update README

Version 0.94
---------
* virtnbdmap: use outputhelper for replay

Version 0.93
---------
* Code cleanup

Version 0.92
---------
* Code cleanup
* virtnbdbackup: Relax check for empty target directory: now only fails if
already an backup (partial or not) exists within the target directory.

Version 0.91
---------
* Code cleanup

Version 0.90
---------
* Code cleanup

Version 0.89
---------
* virtnbdrestore: check if additional boot files such as nvram/kernel images
exist and if not, restore them to the original path instead of just warning
the user about manual steps being required.

Version 0.88
---------
* virtnbdrestore: Allow restore into non-empty directories, so one can
restore the disk files into an existing libvirt managed volume directory.
If the restore target path is an libvirt managed pool, refresh the
pool contents. (#67)
* virtnbdrestore: dont continue with restore if -o dump is specified.

Version 0.87
---------
* Code cleanup
* Add support for remote backup via NBD+TLS (#66)
* Update README

Version 0.86
---------
* libvirt uri: both user and password parameters are mandatory if no qemu+ssh
session is specified.

Version 0.85
---------
* virtnbdrestore: check if target files exist during remote operation and
fail accordingly.
* Remote backup: use paramikos built in sftp client to copy files instead
of third party scp module.

Version 0.84
---------
* Add --nbd-ip option: can be used to bind nbd service for backup task
to specific remote IP.

Version 0.83
---------
* Code reorg
* Fix remote restore: UnboundLocalError: local variable 'pid' referenced
before assignment

Version 0.82
---------
* Move common arguments to separate file.

Version 0.81
---------
* Report PID and errors of commands executed on remote system.
* Initiate ssh session for backup of boot config only if required.
* Update README

Version 0.80
---------
* Fix debug output
* Add remote backup functionality (#65).

Version 0.79
---------
* Add -U/--uri option: can be used to specify libvirt connection URI, if
authfile is specified, use openAuth to authenticate against libvirt daemon.
* Add --user and --password options: can be used to authenticate against
libvirt daemon.
* Update README, add notes about ovirt/rhev etc.

Version 0.78
---------
* Code cleanup
* Update README

Version 0.77
---------
* Code cleanup
* virtnbdbackup
 + detect if there is an active backup operation running and fail accordingly
 instead of running into "Cannot acquire state change lock" timeout exception.

Version 0.76
---------
* virtnbdbackup:
 + Change some log messages: more detailed report on skipped devices during backup.
 + Add option --freeze-mountpoint: during backup, only filesystems of
 specified mountpoints are freezed and thawed. (#60)
* virtnbdrestore:
 + Option -c would only adjust virtual machine config for the first disk.
 + Option -c now correctly removes excluded disks from the adjusted virtual
 machine configuration.
* Add --version option for all utilities.
* Vagrant scripts: test installation/execution after creating rpm packages.
* Update README

Version 0.75
---------
* virtnbdbackup: report amount of thawed/freezed filesystems during backup.
* Update README

Version 0.74
---------
* virtnbdbackup: add some more debug messages around freezing/thwaing
filesystems and backup job operation.

Version 0.73
---------
* virtnbdbackup: limit the amount of concurrent workers to the amount of
disks detected, so users cannot specify an higher amount of workers than
disks are attached to the virtual machine.

Version 0.72
---------
* Code cleanup
* Update README

Version 0.71
---------
* Code cleanup
* Update README

Version 0.70
---------
* Code cleanup
* Exit gracefully if setting up the logfile already fails.
* virtnbdrestore: make option `-a restore` default, if output directory
is named "dump" or `-a dump` is specified, stream information is dumped.
* Update README, examples and tests accordingly

Version 0.69
---------
* Code cleanup
* virtnbdbackup: exit early if removing checkpoints fails during full
backup.
* virtnbdrestore: do not write zeroes during restore of image, as the
resulting zeroed regions are then reported as being "data": this would result
in further backups of the virtual machine to be thick provisioned, as even
zeroed regions are saved. (#56)
* virtnbdrestore: dont adjust vm config if option -c is missing.
* virtnbdrestore: now removes existent backing stores if adjust setting is
enabled: usually the case if virtual machine operated on snapshot during
backup.

Version 0.68
---------
* Code cleanup at different places. Make the output helper wrapping.
* Use Elementree from lxml for easier access via xpath, adjust dependencies
* Adjust pylintrc
* virtnbdrestore: add option -c: adjusts required paths within restored
virtual machine config so it can be defined using virsh. (#54)
* virtnbdrestore: add option -D: in combination with option -c can be used to
register virtual machine on libvirt daemon after restore.
* virtnbdrestore: add option -N: redefine domain with specified name, if not
passed, prefix "restore_" is added.

Version 0.67
---------
* Disable xml based check if incremental backup is enabled for
libvirt versions >= 7.6.0, the feature is enabled by default
now: https://github.com/libvirt/libvirt/blob/master/NEWS.rst#v760-2021-08-02
* Fix dependencies for rpm package: nbdkit-plugin-python3 -> nbdkit-plugin-python
* Fix documentation regards build on almalinux
* virtnbdbackup: introduce backup mode "auto": automatically execute full
backup if target folder is empty. If full backup exists in target folder,
switch to incremental backup mode automatically. (#52)

Version 0.66
---------
* virtnbdmap: use absolute path of files specified.
* nbdkit plugin: add debug flag, be less verbose (#47)
* Fix pylint warnings
* Update README

Version 0.65
---------
* virtnbdmap: open target device with O_DIRECT during replay of incremental
data to speedup the process: no need to sync then.
* virtnbdmap: remove nbdkit logfile after exiting successfully
* virtnbdmap: use pidfile and send signal to correct process. (#46)
* virtnbdmap: check if the passed device starts with /dev/nbd
* virtnbdmap: code cleanup

Version 0.64
---------
* virtnbdmap: minor code cleanup
* virtnbdmap: add progressbar during replay of incremental backups
* virtnbdmap: pass listen port to qemu-nbd command too.

Version 0.63
---------
* virtnbdmap: update epilog examples, add logfile option
* virtnbdmap: add --readonly option.
* virtnbdmap: add --listen-port option: useful if one wants to map multiple
backups on the same system concurrently.
* virtnbdmap: better error handling if nbdkit process fails to start.
* Update README

Version 0.62
---------
* Add script to create virtualenv with required dependencies.
* virtnbdmap now supports mounting of full->incremental (or differential)
chains: the current approach is to use the nbdkit COW (copy on write) filter
to replay the data blocks against the mapped nbd device.

Version 0.61
---------
* Add vagrant scripts
* Print libvirt library version.
* Disable xml based check if incremental backup capability is enabled for
libvirt versions > 8002000: feature is enabled by default. (#4)

Version 0.60
---------
* Add python3-dataclasses to RPM dependencies.
* Update README

Version 0.59
---------
* Add epilog to help output with some example commands
* Update README

Version 0.58
---------
* Slight code improvements
* Skip devices with type "floppy", just as "cdrom" devices.

Version 0.57
---------
* Backup configured kernel/initrd images used for direct boot too.

Version 0.56
---------
* Fix for Issue (#40): redefining checkpoints fails

Version 0.55
---------
* Correct some wrongly caught exceptions
* Update README


Version 0.54
---------
* (#38) Backup virtual machines loader bios and nvram if defined, provide
notice to user during restore that files must be copied manually.
* virtnbdrestore: Fixes for --until option:
	+ would not stop processing further files since --sequence option was introduced
	+ would stop before actually processing specified checkpoint, now function
	stops after restoring data of checkpoint to be reached.
	+ extend tests
* Update README

Version 0.53
---------
* Cleanup codebase
* Fix differential backup for offline domains: apply same logic as during
incremental backup.

Version 0.52
---------
* Update help regarding --qemu option: works with diff/inc now too
* Group possible options in help output
* Introduce more exceptions, cleanup codebase.

Version 0.51
---------
* Cleanup codebase, no functional changes.

Version 0.50
---------
* Fix sequence restore: function returned too early.
* Parameter for --sequence option does not require absolute path anymore.


Version 0.49
---------
* virtnbdrestore now support restoring a manual sequence of data files
passed by the --sequence option.

Version 0.48
---------

* Internal code changes: start using proper exceptions if stream format
is not parseable
* virtnbdbrestore: add --sequence option, allows to specify a comma separated
list of backup files whose information should be dumped. Next step is to
implement restore of a specified sequence.


Version 0.47
---------
* Fix issue #37: If an backup directory contains multiple differential
or incremental backups, its not possible to use the --until option for
point in time restore, because the differential backups defaulted to
the first checkpoint name in the metadata header. Recover would
stop too early in such cases..

Now the checkpoint name in the metadata header contains the same timestamp as
the target files, which make it possible to use the --until option too.

Version 0.46
---------
* Add changelog file

Version 0.45
---------
* Adds differential backup option: backup option -l diff now saves the
delta since the last incremental or full backup. Option -q now uses
nbdinfo instead of qemu-img map, which supports reading specific bitmap
information: now option -q can be used during incremental/differential backup
too,which might be good for debugging issues.
* If incremental backup for offline Domain is attempted, backup would save all
data, not just dirty extents: now saves only dirty extents.  
* Extends the testsuite with tests for the implemented new features
* Fixes various pylint warnings, better error handling in some situations.
* Update documentation and executable dependencies for the debian package build.

**Full Changelog**: https://github.com/abbbi/virtnbdbackup/compare/v0.44...v0.45

Version 0.44
---------
Ensure checkpoint chain consistency:

As discussed during issue #33, the "parent" option to a checkpoint
created is readonly currently. Thus, virtnbdbackup can not force
an parent checkpoint during creation of the actual checkpoints.
    
This means we cannot ensure that the complete checkpoint chain
we are based on for the current incremental backup is entirely
created by us, like so:
    
   > virsh checkpoint-list vm1 --tree
   > virtnbdbackup.0                                                    r
   >   |
   >   +- virtnbdbackup.1
   >       |
   >       +- virtnbdbackup.2
   >           |
   >           +- not-our-checkpoint

The delta for "not our checkpoint" would never be saved because
we dont know about this checkpoint.

Now virtnbdbackup checks for checkpoints which might have been
created by users or third party applications and exits with
error if incremental or full backup is attempted.

**Full Changelog**: https://github.com/abbbi/virtnbdbackup/compare/v0.43...v0.44

Version 0.43
---------
Remove --checkpoint option: has never behaved the way it should. The parent
option to a checkpoint XML definition is, according to the documentation read
only. So currently its not supported to force a specific checkpoint using this
option.

**Full Changelog**: https://github.com/abbbi/virtnbdbackup/compare/v0.42...v0.43

Version 0.42
---------
* Fixes some pylint/flake8 warnings.

**Full Changelog**: https://github.com/abbbi/virtnbdbackup/compare/v0.41...v0.42

Version 0.41
---------
Minor code changes regards checkpoint handling, fix some pylint warnings.

**Full Changelog**: https://github.com/abbbi/virtnbdbackup/compare/v0.40...v0.41

Version 0.40
---------
Introduce custom handler for logging facility to track number of warnings
during backup operation.

Recent changes introduce a new logging facility handler that counts the number
of warnings during backup (for example if qemu agent is not
reachable etc..)

If option "--strict" is set during backup, the exit code will now set to
2.

This allows calling applications or scripts to have a more detailed
error handling in regards to warnings and/or errors.

**Full Changelog**: https://github.com/abbbi/virtnbdbackup/compare/v0.39...v0.40

Version 0.39
---------
Adds full support for offline domains:

* Allows for incremental backup of offline domains: changed data for last
checkpoint will be saved until a new checkpoint is created.  * Adds proper
error handling for cases where starting the NBD server for offline domains
causes error

**Full Changelog**: https://github.com/abbbi/virtnbdbackup/compare/v0.38...v0.39

Version 0.38
---------
Add support for backup of shutdown domains (#27)

Usually virtnbdbackup makes most sense while operating
on running domains. Only if a virtual domain is running,
checkpoints can be defined via libvirt API and the
required backup operations can be executed. The qemu
Process will then start the NBD backend in order to
receive the backup data.

With the recent changes, virtnbdbackup now detects
if the backup source is offline and then:

 * Forces the backup mode to "copy": no full or
 incremental backup supported because we can't create
 an checkpoint via libvirt API. Copy type backup
 is like a full backup just without checkpoint.
 * Forces the amount of workers to 1
 * Starts an NBD Process via qemu-nbd for each disk, operating
 directly on the Virtual Disk files as detected from the VM
 configuration.
 * Queries required extents and data from the NBD
 Service and saves it accordingly.

Full changeset:

**Full Changelog**: https://github.com/abbbi/virtnbdbackup/compare/v0.36...v0.38

Version 0.36
---------
Fix for issue #28
Fix for issue #26 

Only minor changes regards logging messages.

**Full Changelog**: https://github.com/abbbi/virtnbdbackup/compare/v0.35...v0.36

Version 0.35
---------
**Full Changelog**: https://github.com/abbbi/virtnbdbackup/compare/v0.34...v0.35

Version 0.34
---------
Slight improvements to `virtnbdmap`.
Show argument parameter default values in help outputs where useful.  Add
nbdkit and python plugin to rpm/debian package dependencies.

**Full Changelog**: https://github.com/abbbi/virtnbdbackup/compare/v0.33...v0.34

Version 0.33
---------
Slight improvements for `virtnbdmap`:

* add option to pass listening socket for nbdkit process
* add option to specify export name, might be required on older versions

Tests:

* add test for `virtndbmap` (currently skipped on github because access to nbd
devices not possible within docker containers.)

Version 0.32
---------
Introduce new `virtnbdmap` utility to allow simple single file and instant recovery, see:

 https://github.com/abbbi/virtnbdbackup#single-file-restore-and-instant-recovery

Version 0.31
---------
Add documentation and update for nbdkit plugin: Its now possible to create
single file restores or do instant recovery by booting the virtual machine
from the thin provisioned full backups.

Version 0.30
---------
Various bugfixes, added some small features in `virtnbdrestore`: it now
attempts to use the original disks name for the disk files and copies the
virtual machine config file. Fixed a few pylint warnings as well.

Updated readme and improved testsuite.

**Full Changelog**: https://github.com/abbbi/virtnbdbackup/compare/v0.29...v0.30

Version 0.29
---------
Version adds support for concurrent backup of multiple disks:

 https://github.com/abbbi/virtnbdbackup#backup-concurrency

**Full Changelog**: https://github.com/abbbi/virtnbdbackup/compare/v0.28...v0.29

Version 0.28
---------
Minor changes: dont attempt to re-create checkpoints during copy backup,
changes in error handling.

Version 0.27
---------
Minor bugfix: exit gracefully if backup to stdout is attempted in raw format:
does not work with the new zip archive stream.

Version 0.26
---------
Implement feature from issue #16: it is now possible to dump complete backup
output to stdout.

Version 0.25
---------
Fixes issue #17: Fails to restore missing checkpoints after the 11th backup


Version 0.24
---------
* Small change for log format output, enclose date.
* Code reformat.

Version 0.23
---------
No real functional changes:

 * Reformat python code with python-black to have a common code format

Version 0.22
---------
* Support compression with `--compress` option: blocks saved in stream are lz4
compressed, add test cases.
* Fix checkpoint creation with `--include` option: do not create checkpoints
for disks which are not part of include list * Various other small Bugfixes
related to error handling
* If backup is done to stdout, save virtual machine config to current folder.
* Update README

Version 0.21
---------
* Add support for backup of raw disk images during full backup using option
`--raw`:

Version 0.20
---------
* Add Support for backup of transient domains in cluster environments.
* Update README

Version 0.19
---------
* If libvirthelper failed to setup backup job because of exception, an already
freezed guest filesystem was not thawed.
* Fix tests.

Version 0.18
---------
* During backup to stdout it was attempted to close non opened file handle
resulting in backup error
* Include used libnbd version in backup log
* Fix some pylint warnings
* Update README

Version 0.17
---------
* Remove sh module from requirements, obsoleted by subprocess module

Version 0.16
---------
* Code cleanup, fix pylint warnings
* Update README

Version 0.15
---------
* Minor code changes, move some functions to common class
* Use makedirs for targetfolder to support nested target paths.
* Updated README with more information about backup workflow.

Version 0.14
---------
Code cleanups
Change shebang so executables appear in process list with real name instead of
python3 executable

Version 0.13
---------
Write backup to partial file first

In case backup utility receives signal or fails otherwise,
the backup data is written to the regular target file and
it is assumed everything is OK.

Now virtnbdbackup writes the data to a partial file first
and renames the file as last step. During incremental
backup the target directory is checked for possible existence
of partial backups and backup is aborted with error.

Version 0.11
---------
Mostly code cleanup and pylint related warning fixes.

Version 0.10
---------
* Allow multiple concurrent backups as NBD server is now connected via local
unix domain socket instead of TCP, allowing unique socket file names
* Remove dependency on sh module

Version 0.8
---------
* Fix name in setup.py
* Provide RPM package for download

Version 0.7
---------
* Minor code changes, improved error and signal handling
* Update README with common backup errors
* Introduce and use __version__, show version in log and command output

Version 0.5
---------
* Show progress during restore, be less verbose.

Version 0.4
---------
* Add per disk progress bar

Version 0.3
---------
* backup: now calls fsFreeze() and fsThaw() functions to ensure consistent
filesystems during backup.

Version 0.2
---------
* Fix exception in virtnbdrestore due to missing arguments
* Scratchfile target file name is now more unique, not causing issues if
multiple domains are backed up at the same time
* Minor tweaks and improvements

Version 0.1
---------
First release version with following features:

* Supports Full/copy/inc backup of virtual machines
* Skips disks and direct attached disks which do not support backup via changed block tracking
* Creates logfile for each executed backup
* Allows to manually exclude certain disks for backup