o z}a* @sddlmZddlmZeddlmZddlmZddlmZddlm Z ddl Z ddl Z ddl Z ddl Z ddlZddlZddlZddlmZdd lmZdd lmZdd lmZdd lmZdd lmZddlmZddlmZddlmZddlmZddlmZddlmZddlmZddlmZddlm Z ddlm!Z!ddlm"Z"ddl#Zddl#m$Z$ddl%mZddl&m&Z&da'ddZ(dVddZ)d d!Z*d"d#Z+d$d%Z,d&d'Z-d(d)Z.d*d+Z/d,d-Z0d.d/Z1d0d1Z2d2d3Z3d4d5Z4d6d7Z5d8d9Z6d:d;Z7dd?Z9d@dAZ:dBdCZ;dDdEZdJdKZ?dLdMZ@ejAfdNdOZBGdPdQdQeZCdRdSZDdTdUZEdS)W)print_function)standard_library)map)next)object)rangeN) __version__)asyncscheduler) commandline)diffdir)dup_collections)dup_temp)dup_time) file_naming)config)gpg)logmanifest)patchdir)path)progress)tempdir)util)BadVolumeException)datetimecCs6ddl}ddl}tjjdkr||d}||S)Nrreplace)getpasslocalesys version_infomajorencodegetpreferredencoding)messagerrr&4/usr/lib/python3/dist-packages/duplicity/dup_main.py getpass_safeHs   r(FcCsPz|r tjdWStjdWStyYnw|r;tjjtjjvs*tjjtjjvr;dtjvr;t t dtjdS|s^tjjtjjvsMtjjtjjvr^dtjvr^t t dtjdStj rdtj rfdS|dvrldS|dkrtjjsxtjjrtjjrtj s|sdS|dkrtjjstjjrtjjrtj s|sdStt d d } |r|d kr|rtjj}n.tjj}n)|r|rtjjrtjj}ntt d d}n|rtjjrtjj}ntt dd}|d kr|}n|rtt d}ntt d}||kstjt dtjd dd}q|s&tjjs&tjjs&|s&tjt dtjd dd}q|S)a^ Check to make sure passphrase is indeed needed, then get the passphrase from environment, from gpg-agent, or user If n=3, a password is requested and verified. If n=2, the current password is verified. If n=1, a password is requested without verification for the time being. @type n: int @param n: verification level for a passphrase being requested @type action: string @param action: action to perform @type for_signing: boolean @param for_signing: true if the passphrase is for a signing key, false if not @rtype: string @return: passphrase SIGN_PASSPHRASE PASSPHRASEz.Reuse configured PASSPHRASE as SIGN_PASSPHRASEz.Reuse configured SIGN_PASSPHRASE as PASSPHRASE)collection-status list-currentremove-all-but-n-fullremove-all-inc-of-but-n-full remove-oldfullincz)PASSPHRASE variable not set, asking user.Trz!GnuPG passphrase for signing key: z GnuPG passphrase for decryption:z.Retype passphrase for signing key to confirm: z-Retype passphrase for decryption to confirm: z=First and second passphrases do not match! Please try again. force_printFzICannot use empty passphrase with symmetric encryption! Please try again.)osenvironKeyErrorr gpg_profilesign_key recipientshidden_recipientsrNotice_ encryption use_agentrestartInfosigning_passphrase passphraser(LogWARNING)naction for_signing use_cachepass1pass2r&r&r'get_passphraseRs                   rNcCs>z t|r t|sWn tyYnwtdtjjdS)z Fake writing to backend, but do go through all the source paths. @type tarblock_iter: tarblock_iter @param tarblock_iter: iterator for current tar block @rtype: int @return: constant 0 (zero) Nr)r StopIterationrProgressr statsSourceFileSize) tarblock_iterr&r&r' dummy_backups  rTc Cstjj}tjj}zKt|}|rQ|j|kr&|s|jsWdS|r&|j|kr&WdS|j|krHtt dt |t |jftj j ||WdSt|}|sWdSWdStyrtt dt |t |jftj j YdSw)ay Fake writing to backend, but do go through all the source paths. Stop when we have processed the last file and block from the last backup. Normal backup will proceed at the start of the next volume in the set. @type tarblock_iter: tarblock_iter @param tarblock_iter: iterator for current tar block @rtype: int @return: constant 0 (zero) z>File %s complete in backup set. Continuing restart on file %s.z=File %s missing in backup set. Continuing restart on file %s.N)rrB last_index last_blockrprevious_indexprevious_blockrWarnr?ruindex ErrorCoderestart_file_not_foundqueue_index_datarO)rSrUrV iter_resultr&r&r'restart_position_iterators6         r_csdd}fddfddfdd}tjs'd }tj|d }|nKtjj}tj|tj|tj sGtj sG|tjj|nt t d ||_tjj} t t d tjjttjjtjjftjj}t|d } d } tjrtj|d tjtjd ksJttj} g} | sD||d 7}tj |tj!tj"d}t#$t%|}tj!rt&'||j(tj)tj*} ntj"rt&+||j(tj*} n t&,||j(tj*} |-t.}|j/|g||R|0dt&1d||2||d kr|3|3n|4|4| 5| 6fdd|||ft 7t d|t8j9j:tjr6tj;|tj<|ksBJd|| r| D]}| |7} qF|=t8j9>| S)a Encrypt volumes of tarblock_iter and write to backend backup_type should be "inc" or "full" and only matters here when picking the filenames. The path_prefix will determine the names of the files written to backend. Also writes manifest file. Returns number of bytes written. @type backup_type: string @param backup_type: type of backup to perform, either 'inc' or 'full' @type tarblock_iter: tarblock_iter @param tarblock_iter: iterator for current tar block @type backend: callable backend object @param backend: I/O backend for selected protocol @rtype: int @return: bytes written cSs\|\}}|durd}d}|r|d8}|\}}|dur"|}|}|r(|d8}||||fS)z3Return start_index and end_index of previous volumeNr&r3) recall_indexget_previous_index)rS start_index start_block end_index end_blockr&r&r' get_indicies%s   z$write_multivol..get_indiciesc s|g|}|d}|durdStdtjdD]4}|g|}|d}||kr.n!|dur5dSttdt|t ||ft d|q||krodt |||f}t tdt |tjj|dSdS)Nsizer3zD%s Remote filesize %d for %s does not match local size %d, retrying.rz%s %d %dz$File %s was corrupted during upload.) query_inforr num_retriesrr>r?rnowrescapetimesleep FatalErrorfsdecoder[volume_wrong_size) orig_size dest_filenameinforgattempt code_extra)backendr&r'validate_block5s* z&write_multivol..validate_blockcs:|}tj|kr|||||jr||S)z Retrieve file size *before* calling backend.put(), which may (at least in case of the localbackend) rename the temporary file to the target instead of copying. )getsizer skip_volumeputstatdelete)tdprrvol_numputsize)rvrwr&r'rzIs   zwrite_multivol..putcs~tjjstjjrtjjsdStjdtjtjd}||j dkr*t t dt j jtjr=ttj||jd}|dSdS)a When restarting a backup, we have no way to verify that the current passphrase is the same as the one used for the beginning of the backup. This is because the local copy of the manifest is unencrypted and we don't need to decrypt the existing volumes on the backend. To ensure that we are using the same passphrase, we manually download volume 1 and decrypt it with the current passphrase. We also want to confirm that we're using the same encryption settings (i.e. we don't switch from encrypted to non in the middle of a backup chain), so we check that the vol1 filename on the server matches the settings of this run. Nr3 encryptedgzippedzQRestarting backup, but current encryption settings do not match original settings)rr:r<r=r;rgetr@ compressionvolume_name_dictrrnr?r[enryption_mismatchrestore_get_enc_fileobjrvvolume_info_dictclose) backup_setr vol1_filenamefileobj) backup_typer&r'validate_encryption_settingsWs$   z4write_multivol..validate_encryption_settingsrfhz:Skipping encryption validation due to glacier/deep storagez-Restarting after volume %s, file %s, block %sr3rSHA1cs |||SNr&)r}rrr~)rzr&r's z write_multivol..zProcessed volume %dz)Forced assertion for testing at volume %d)?rrBrManifest set_dirinfo last_backupget_local_manifest checkManifest setLastSaveds3_use_deep_archives3_use_glacierrrYr?rrVr> start_volrrZrUr_rtrackerset_start_volumeprogress_threadstartasync_concurrencyr AsyncSchedulerremember_next_indexrrr@rr new_tempduppathparser GPGWriteFilenamer:volsize GzipWriteFilePlainWriteFilesetdata VolumeInfoset_infoset_hashget_hashadd_volume_info to_partialflushappend schedule_taskrPr rQrRsnapshot_progressfail_on_volumeset_files_changed_infoget_delta_entries_file)rrS man_outfp sig_outfprvrfrr~mfrVat_end bytes_written io_scheduler async_waitersrrr}viwaiterr&)rvrrzrwr'write_multivols  !              4rcCsZ|dks |dks Jtj|ddd}tj|dd}tj|dtjd}ttj|||}|S)a5 Return a fileobj opened for writing, save results as manifest Save manifest in config.archive_dir_path gzipped. Save them on the backend encrypted as needed. @type man_type: string @param man_type: either "full" or "new" @rtype: fileobj @return: fileobj opened for writing r1r2T)rpartialr)rr)rrrr@r get_fileobj_duppatharchive_dir_path)rpart_man_filenameperm_man_filenameremote_man_filenamerr&r&r'get_man_fileobjs$ rcCsX|dvsJtj|ddd}tj|dd}tj|tjtjd}tjtj|||dd}|S)a; Return a fileobj opened for writing, save results as signature Save signatures in config.archive_dir gzipped. Save them on the backend encrypted as needed. @type sig_type: string @param sig_type: either "full-sig" or "new-sig" @rtype: fileobj @return: fileobj opened for writing full-signew-sigFT)rr)rr) overwrite)rrrr@rr rr)sig_typepart_sig_filenameperm_sig_filenameremote_sig_filenamerr&r&r'get_sig_fileobjs$  rcCs$tjr#tt_ttj}t|tjtj dt t t_ tjr7ttj}t|}|jddnStd}td}ttj|}td|||tj}||||||tjrdtj _tj tddtjjtjtjjd|jddttj |dS) z Do full backup of directory to backend, using archive_dir_path @type col_stats: CollectionStatus object @param col_stats: collection status @rtype: void @return: void TNsig_chain_warningrr1Y@rF) rrProgressTrackerrr DirFullselectrT set_evidencerQr set_selectionLogProgressThreadrdry_run set_valuesrrDirFull_WriteSigrrvr to_remoteto_finalfinishedjoinrTransferProgresstotal_bytecounttotal_elapsed_secondsspeedprint_statistics) col_statsrSrrrr&r&r' full_backup!sF      rcCs@|jstjrttdtjjdSttddS|jdS)z Get last signature chain for inc backup, or None if none available @type col_stats: CollectionStatus object @param col_stats: collection status zdFatal Error: Unable to start incremental backup. Old signatures not found and incremental specifiedz.No signatures found, switching to full backup.Nr) matched_chain_pairr incrementalrrnr?r[inc_without_sigsrYrr&r&r'check_sig_chain[s  rcCs8tjr|tj_tjtd}tj|tj dddSdS)zp If config.print_statistics, print stats after adding bytes_written @rtype: void @return: void zBackup StatisticsTr5N) rrr rQTotalDestinationSizeChangeget_stats_logstringr?rrFNOTICE)rQr logstringr&r&r'rms rcCsbtjs"t|jtjtjkr"tdt tjtjks"Jdtj rHt t _ t tj|}t|t j t jdtt t _tjrYt tj|}t|}nPtd}td}t tj||}td|||tj}||| ||| tj rdt j_!t j"t#$ddt j j%t j &t j j'dt(t j|d S) zs Do incremental backup of directory to backend, using archive_dir_path @rtype: void @return: void rzBtime not moving forward at appropriate pace - system clock issues?Frr2TrrN))rrBr setprevtimeend_timecurtimeprevtimerlrm setcurtimerrrr DirDeltar get_fileobjsrTrrQr rrrrrrDirDelta_WriteSigrrvrrrrrrrrrrr) sig_chainrSr new_sig_outfp new_man_outfpr&r&r'incremental_backupzs\       rcCstjptj}||}t||}|D]5}|jdkrJdt | t | f}dt| t | |jf}t|tjtjj|dqdS)z List the files current in the archive (examining signature only) @type col_stats: CollectionStatus object @param col_stats: collection status @rtype: void @return: void deleted%s %sz%s %s %sTN)r restore_timerrget_signature_chain_at_timer get_combined_path_iterrdifftype timetoprettygetmtimerroget_relative_path timetostringrktyperrFINFOInfoCode file_list)rrlr path_iterr user_infolog_infor&r&r' list_currents"    rcCsjtjr t|dSttjt|s3tjr'tt dt tjtj j dStt dtj jdSdS)z Restore archive in config.backend to config.local_path @type col_stats: CollectionStatus object @param col_stats: collection status @rtype: void @return: void Nz,%s not found in archive - no files restored.z-No files found in archive - nothing restored.)rrrestore_get_patched_rop_iterr Write_ROPaths local_path restore_dirrrnr?rror[restore_dir_not_foundno_restore_filesrr&r&r'restores     rc s$tjr ttjdndtjptj}||}|s J|j| |}d|D]}t |7q)dgfdd}t tj dsFtj r}g}|D]}|}|} | D] } ||j| qWqJtj rwtddd d |Dd Stj |tt||} tttj| } t| S) z Return iterator of patched ROPaths of desired restore data @type col_stats: CollectionStatus object @param col_stats: collection status /r&rc 3s|}|}|D]A}zt|j|j||j|VWnty3}z |VWYd}~nd}~wwdd7<tt ddfdq dS)z.get_fileobj_iterpre_process_download_batchzRequired volumes to restore: z css|]}|VqdSr)decode).0 file_namer&r&r' sz/restore_get_patched_rop_iter..N)rrtuplesplitrrrget_backup_chain_at_timeall_backup_chainsget_sets_at_timelenhasattrrvrrrrrrr>rr"listrrTarFile_FromFileobjstarfiles2rop_iter) rrl backup_chainbackup_setlistsr! file_namesrrrr~ fileobj_iterstarfilesr&rr'rs:      rc Cst|}t|}||| t||\}}}|sjdtd|dt|td|td|df}t j rat j dt|} tj|tjtjjdttd | jjt| f| tj|tjjd|d } |jrzt jjrzt| | S) aH Return plaintext fileobj from encrypted filename on backend If volume_info is set, the hash of the file will be checked, assuming some hash is available. Also, if config.sign_key is set, a fatal error will be raised if file not signed by sign_key. with --ignore-errors set continue on hash mismatch z%s %s %s %s z)Invalid data - %s hash mismatch for file:rzCalculated hash: %szManifest hash: %sr3zHash mismatch for: %s)codez;IGNORED_ERROR: Warning: ignoring error as requested: %s: %srb)rrr rrrestore_check_hashr?rror ignore_errors duplicityerrorsrrrFERRORr[mismatched_hashrY __class____name__uexcrnfiltered_open_with_deleterr:r;restore_add_sig_check) rvfilename volume_info parseresultsr}verified hash_paircalculated_hash error_msgexcrr&r&r'r!s4     rcCs>|}|rt|d|}||dkrd||fS d||fS)z Check the hash of vol_path path against data in volume_info @rtype: boolean @return: true (verified) / false (failed) rr3FT) get_best_hashrr)rEvol_pathrHrIr&r&r'r9Hs   r9cs<ttjr tjtjsJfdd}|dS)zo Require signature when closing fileobj matches sig in gpg_profile @rtype: void @return: void csj}|dur dn|}tjj}|durdn|}tt|t| }||d||dkrGtt d||d||dftj j dSdS)z"Thunk run when closing volume fileNNonez#Volume was signed by key %s, not %s) r get_signaturerr:r;minr,rrnr?r[unsigned_volume) actual_sigr;ofsrr&r'check_signaturebs  z.restore_add_sig_check..check_signatureN) isinstancer FileobjHookedrrGPGFileaddhook)rrUr&rTr'rCXs    rCcCstt|tj}d}d}|D]#\}}|st|j}|s#t|j}||tj s.|d7}|d7}qt t dt dd||t dd||f|dkrRdadSdS) z Verify files, logging differences @type col_stats: CollectionStatus object @param col_stats: collection status @rtype: void @return: void rr3zVerify complete: %s, %s.z%d file comparedz%d files comparedz%d difference foundz%d differences foundN)r collate2itersrrrrROPathrcompare_verbose compare_datarr>r?ngettextexit_val)rcollated diff_count total_count backup_ropath current_pathr&r&r'verifyqs:      rec Cs|\}}||}|sttddSdttj|}tj rUt t ddt |d|tj sQ|j||D]}z tj|Wq;tyPYq;wdSdSt t ddt |d|dtddS) z Delete the extraneous files in the current backend @type col_stats: CollectionStatus object @param col_stats: collection status @rtype: void @return: void z6No extraneous files found, nothing deleted in cleanup.N z Deleting this file from backend:z"Deleting these files from backend:z#Found the following file to delete:z$Found the following files to delete:z?Run duplicity again with the --force option to actually delete.)get_extraneousrrYr?rrrrorforcer>r^r,rrvr|rr Exception)r ext_local ext_remote extraneousfilestrfnr&r&r'cleanupsJ    rocCs(tjdusJ|tjt_t|dS)z Remove backup files older than the last n full backups. @type col_stats: CollectionStatus object @param col_stats: collection status @rtype: void @return: void N)r keep_chainsget_nth_last_full_backup_time remove_time remove_oldrr&r&r'remove_all_but_n_fulls  rtcCstjdusJdd}dd}|tj}|r'tdtd||tdf|jr:|jd jtjkr:ttd |tj}tj rLt d d |D}|sWt td dStj rt t ddt|d||||tj7}||D]9}tj rt|tjrtd}ntd}nt|tjrtd}ntd}t |t|jtjs|jtj dqy|jdddSt t ddt|d||dtddS)z Remove backup files older than config.remove_time from backend @type col_stats: CollectionStatus object @param col_stats: collection status @rtype: void @return: void NcSddd|DS)z.Return string listing times of sets in setlistrfcSsg|] }t|qSr&)rrget_timer$r3r&r&r' sz5remove_old..set_times_str..r)setlistr&r&r' set_times_strz!remove_old..set_times_strcSru)z2Return string listing times of chains in chainlistrfcSsg|]}t|jqSr&)rrrrwr&r&r'rxz7remove_old..chain_times_str..ry) chainlistr&r&r'chain_times_strr|z#remove_old..chain_times_strz%s %s %sz#There are backup set(s) at time(s):z9Which can't be deleted because newer sets depend on them.r3zCurrent active backup chain is older than specified time. However, it will not be deleted. To remove all your backups, manually purge the repository.css8|]}t|tjr |jst|tjr|jr|VqdSr)rVr SignatureChaininclist BackupChain incset_list)r$xr&r&r'r&s  zremove_old..z*No old backup sets found, nothing deleted.zDeleting backup chain at time:z Deleting backup chains at times:rfz5Deleting any incremental signature chain rooted at %sz2Deleting any incremental backup chain rooted at %sz$Deleting complete signature chain %sz!Deleting complete backup chain %s) keep_fullrz-Found old backup chain at the following time:z/Found old backup chains at the following times:z5Rerun command with --force option to actually delete.)rrrget_older_than_requiredrrYr?rrget_chains_older_than!remove_all_inc_of_but_n_full_moder.r>rhr^r,get_signature_chains_older_thanreverserVr rrrrr|r)rr{rreq_listr~chain chain_descr&r&r'rssp         rsc sd}tjptj}ttjd|jdd}ttjd|jdd}tj }tj }|j d|dd}|j d|dd}t |ddd t |d dd |sWt td dS|D]zt fd d |Dd}Wn tyud}Ynw|rt ttj|ng} D]} t| } | jp| j|ksqz| | t td| fWqtyYnw| jdkrt| jt| jp| jt td| ftj| } tj| jtjtj d} t!"t| }|j#dd}t$%| || &|&tj'|| |(qqY|j)|dd}|j)|dd}t |ddd t |ddd |D]2zt fdd |Dd}Wn tyKd}Ynw|rS|*ng}*D]}|+|kseqYz||t td|j,fWqYtyYnw|jdkrt|jt|+|-}tj|jdd}t!"t|}t.j/|j#ddd}t |j01D]c\}} t td| ft2tj| |j3|} tj|j|tjtj d} t!"t| }|j#dd}t$%| || &|&tj'|| t44|j3|}|5dt67d||8||(q|j9&|j:dd}tj|jdtjtj d}t!"t|}|j#dd}t$%|||&|&tj'|||(qYq-tj&tj&dS) z Replicate backup files from one remote to another, possibly encrypting or adding parity. @rtype: void @return: void replicateNrF)localfilelistrcS|jSr start_timerr&r&r'r%zreplicate..)keycSrrrrr&r&r'r&rzNo old backup sets found.cg|] }|jjkr|qSr&rr$r src_chainr&r'rx,zreplicate..zSignature %s already replicatedrzReplicating %s.rwb)mode) filename_listcSrrrrr&r&r'rKrcSrrrrr&r&r'rLrcrr&rrrr&r'rxOrzBackupset %s already replicatedr2Trrrr8)rrr);rrrrr CollectionsStatus src_backendrrvr.get_signature_chainssortedrr>r? IndexErrorrrr get_filenamesrlrremoverC ValueErrorr rrrget_fileobj_readrr@rr r filtered_openr copyfileobjrrzr|get_backup_chains get_all_setsrvremote_manifest_nameget_remote_manifestrrritemsrrcopyrrrrrrB)rIrl src_stats tgt_statssrc_listtgt_list src_chainlist tgt_chainlist tgt_chaintgt_sigssrc_sig_filenamesrc_sigrrDr}tmpobjtgt_setssrc_setrmf mf_filenamemf_tdprir mf_fileobjmf_final_filename mf_final_tdpmf_final_fileobjr&rr'rs                            0rcsgdfdd}fdd}ddfdd d d }fd d }tj}||\}}}tj} || \} } } t| } t|}g}g}|D]}|| vrd|| vrd||rd|||qO| D]}||vsq|| vrx|| |qg|s|stt ddS| | tj stt d|r|s| rt ddtj _|D]}||qttjdrtj||D]}||qdS|rtt dddttj||rtt dddttj|dSdS)z Synchronize local archive manifest file and sig chains to remote archives. Copy missing files from remote to local as needed to make sure the local archive is synchronized to remote storage. @rtype: void @return: void )s.gs.gpgs.zs.gzs.partcstjdkrdStjdksJt|}z tjptj}Wn tj y)YdSw|j dur:|j dur:|j }}n|j }|j }||j koI||j kS)zIndicates if the metadata file should be synced. In full sync mode, or if there's a collection misbehavior, all files are needed. Otherwise, only the metadata for the target chain needs sync. r1TrN) rmetadata_sync_moderrr)rrrr CollectionsErrorrrrl)rDparsed target_chainrrrr&r' is_neededs$     zsync_archive..is_neededcs~i}i}d}|D]1}t|}|sq|jrd}|jdvs|jr9tj|\}}|vr-|}|jr5|||<q|||<q|||fS)a] Return metafiles of interest from the file list. Files of interest are: sigtar - signature files manifest - signature files duplicity partial versions of the above Files excluded are: non-duplicity files @rtype: list @return: list of duplicity metadata files FTr) rrrr rr7rsplitextr)r metafilespartialsneed_passphrasernprbaseextsuffixesr&r' get_metafiless$    z#sync_archive..get_metafilescSsFt|d} z|j}Wn tyYnw||q|dS)z7 Copy data from src_iter to file at fn rTN)open__next__datarOwriter)src_iterrDfilerr&r&r'copy_raws    zsync_archive..copy_rawcsHt|}tj|\}}|vr|}td|j }||}|||fS)zB @return: (parsedresult, local_name, remote_name) F)rrr7rr get_suffixr)rnrrrsuffixloc_namerr&r'resolve_basenames  z&sync_archive..resolve_basenamec Sstj|j}ttdt|z t t j |WdSt yA}zt tdt|t|fWYd}~dSd}~ww)Nz1Deleting local %s (not authoritative at backend).zUnable to delete %s: %s)rrrrrr>r?rroignore_missingr7unlinkrirYrA)rndel_namerr&r&r' remove_locals z"sync_archive..remove_localcsGdddtGfdddt}ttdt||\}}}tj|}||}t t |}|j rA||jn tj||jtjd||tj|dS)z5 Copy remote file fn to local cache. c@seZdZdZddZdS)z2sync_archive..copy_to_local..Blockz; Data block to return from SrcIter cS ||_dSr)r)selfrr&r&r'__init__ z;sync_archive..copy_to_local..Block.__init__N)r@ __module__ __qualname____doc__rr&r&r&r'Blocks rcs4eZdZdZddZfddZddZdd Zd S) z4sync_archive..copy_to_local..SrcIterzG Iterate over source and return Block of data. cSrrrT)rrr&r&r'rrz=sync_archive..copy_to_local..SrcIter.__init__c sz |j|}Wn1ty=t|jdr&|jj}t|dr%|j}nd}ttdt |t ftj jYnw|jsH|jt|S)NrzFailed to read %s: %s)rread get_read_sizerir-rrrnr?rror exc_infor[genericrrrO)rresrrr&r'r s&     z=sync_archive..copy_to_local..SrcIter.__next__cSdS)Nir&rr&r&r'rzBsync_archive..copy_to_local..SrcIter.get_read_sizecSr)Nr&rr&r&r' get_footerrz?sync_archive..copy_to_local..SrcIter.get_footerN)r@rrrrrrrr&rr&r'SrcIters   rzCopying %s to local cache.)rgN)rrr>r?rrorrvrr rrrrrrrr maxsizermoverr)rnrrrrem_namerrr})rrrr' copy_to_locals! z#sync_archive..copy_to_localz;Local and Remote metadata are synchronized, no sync needed.z/Synchronizing remote metadata to local cache...r3syncr"z3Sync would copy the following from remote to local:rfz5Sync would remove the following spurious local files:N)rrvr.rlistdirkeysrrr>r?sortrrNr:rEr-r"rrrrro)rrrrrremlistremote_metafilesignored rem_needpassloclistlocal_metafileslocal_partials loc_needpass local_keys remote_keys local_missinglocal_spuriousrrnr&)rrrrr' sync_archivesf    <            r cCs8|jsJ|jd}tj ptjj}|j|ddS)z Check consistency and hostname/directory of last manifest @type col_stats: CollectionStatus object @param col_stats: collection status @rtype: void @return: void ) check_remoteN)r*get_lastrr@r:rEcheck_manifests)rlast_backup_setr r&r&r'check_last_manifestks rc Cs`|dvrt\}}t|tjj|tjjdd}zt |}Wnt y;t t dt jjYnw|j|j}tjdtjtdtj}||krdt t d||ft jjn t t d||fz ttj\}}Wntjyt t d t jjYnwtd d ||fD} | d krt t d | ft jjdSdSdS)z Check for sufficient resources: - temp space for volume build - enough max open files Put out fatal error if not sufficient to run @type action: string @param action: action in progress @rtype: void @return: void )r1r2rNz!Unable to get free space on temp.r3g333333?z4Temp space has %d available, backup needs approx %d.z1Temp has %d available, backup will use approx %d.zUnable to get max open files.cSsg|]}|dkr|qS)r r&)r$lr&r&r'rxr}z#check_resources..iz_Max open files of %s is too low, should be >= 1024. Use 'ulimit -n 1024' or higher to correct. )rdefaultmkstempr7rrseprr(statvfsrirrnr?r[get_freespace_failedf_frsizef_bavailrrrintnot_enough_freespacerCresource getrlimit RLIMIT_NOFILEerrorget_ulimit_failedrPmaxopen_too_low) rItempfiletempnametempfsrQ freespace needspacesofthardmaxopenr&r&r'check_resources|sP            r+cCstd|tdt|ddtjD}tdd||tdt|tdtjp4tjtj f|td|dS) z4 log Python, duplicity, and system versions zP================================================================================z duplicity %scss|]}t|VqdSr)rro)r$argr&r&r'r&sz$log_startup_parms..zArgs: %sr4rN) rrFrr argvrplatformuname executableversion) verbosityu_argsr&r&r'log_startup_parmss r4c@s0eZdZdZddZddZddZdd Zd S) Restartzo Class to aid in restart of inc or full backup. Instance in config.restart if restart in progress. cCs8d|_d|_d|_d|_d|_d|_||_||dSr)r rrrrUrVrsetParmsrrr&r&r'rszRestart.__init__cCsD|jr d|_|j|_n d|_|j|_|j|_tt|dd|_dS)Nr1r2r3r)rlr rrmaxr,rr7r&r&r'r6s zRestart.setParmscCst|j}||jks|r|jsz|jdkr.ttd|jt t j dt j tj dS||jdkr[ttd|jd||jdft |jd|dD]}||qQdSttd||jf|jt t j dt j tj dSdS)NrzRESTART: The first volume failed to upload before termination. Restart is impossible...starting backup from beginning.zgRESTART: Volumes %d to %d failed to upload before termination. Restarting backup at volume %d.r3zRESTART: Impossible backup state: manifest has %d vols, remote has %d vols. Restart is impossible ... duplicity will clean off the last partial backup then restart the backup from the beginning.)r,rrrr>r?rr|r7execver r-r8rdel_volume_info)rrmf_lenvolr&r&r'rs&       zRestart.checkManifestcCs$|j|j}|j|_|jpd|_dS)Nr)rrrdrUrerV)rrrr&r&r'rs zRestart.setLastSavedN)r@rrrrr6rrr&r&r&r'r5s   r5cCsdtjvrttdtjjtdkr#ttt t t t tjdd}tjtjjdt_tjtjt_ttdtjtjjdd setd tjjttd z t |Wt!"dSt!"w) z Start/end here PYTHONOPTIMIZEz PYTHONOPTIMIZE in the environment causes duplicity to fail to recognize its own backups. Please remove PYTHONOPTIMIZE from the environment and rerun the backup. See https://bugs.launchpad.net/duplicity/+bug/931175 rr3NslockfilezAcquiring lockfile %sF)blockingzJAnother duplicity instance is already running with this archive directory r)#r7r8rrnr?r[pythonoptimize_setgeteuidsetuidsetgidgetegidrrr ProcessCommandLiner r-rrrrrlockpath fasteners process_lockInterProcessLocklockfileDebugacquire user_errorshutdownexit do_backuprrelease_lockfile)rIr&r&r'mains,     rQcCsztjr ttjntttjt|t tj tj | }|dvr*t | |dvr|}|s6nS|}|jr|dvrnt|t_tjj}|dkrVttjjnttjjttjjttd|nttd||t tj tj | }q*n |}|dkrttd d t|nttd tjs|d krtjdur|tjkrttd d}t|td|tj _!|dkrt"|n|dkrt#|n|dkrt$|n|dkrtj%st|dnt&|tj%dn|dkr t'|n|dkrt(|n|dks|dkr"t)|n|dkr,t |nz|dkr5t*nq|d ksC|dksCJ|tj j+rPtd|dtj _,tj j-sytj j.sytd|tj _!tj j,rytj j!tj j,kryt/tdtj0j1|dkrt2|n#t3|}|st2|ntjs|j4rtd|tj _!t5|t6|tj 7t8t9durt:;t9dSdS)N)r,r.r/r0rT)r1r2ro)r1r2r1z.Last %s backup left a partial set, restarting.z7Cleaning up previous partial %s backup set, restarting.rzLast full backup date:r4zLast full backup date: noner2z0Last full backup is too old, forcing full backupr3rrer-r,ror0r.r/rrrz]When using symmetric encryption, the signing passphrase must equal the encryption passphrase.)r?r|get_last_full_backup_timerfull_force_timePrintCollectionStatusrNr:rErrer file_changed PrintCollectionFileChangedStatusrorsrtrr;rDr<r=rnr[rLrrr*rrrrMr_r rN)rIrlast_full_chainrlast_full_timerr&r&r'rO&s                      rO)F)F __future__rfuturerinstall_aliasesbuiltinsrrrrrrFr7r.rr rlr;rr r r r r rrrrrrrrrrrduplicity.errorsrduplicity.configrr_r(rNrTr_rrrrrrrrrrrr9rCrerortrsrr rr+r r4r5rQrOr&r&r&r's                           ,S: ?7'"#Mtd3 < .