o ~_:@sddlmZddlmZddlmZddlZddlZddlZ ddl m Z ddl m Z ddl m Z mZddl mZdd l mZd Zd d ZGd dde jjZdS))division)str)ThreadPoolExecutorN)config)log)FatalBackendExceptionBackendException)progress)utilz2.1.1c Cs0zJddlm}t|dsJzddlm}ddlm}d}|}Wn ty.d}d}YnwtjrA|r8|}nt dtj j n|rG|}nd}Wnty]t d t tj j Ynw|jsj|jtj d }n|d kspJ|j|j|jtj d }t|d r|durt dtj j|S||_|Std)Nr) S3Connectionlookup)OrdinaryCallingFormat)SubdomainCallingFormatTFzUse of new-style (subdomain) S3 bucket addressing wasrequested, but does not seem to be supported by the boto library. Either you need to upgrade your boto library or duplicity has failed to correctly detect the appropriate support.z_This backend (s3) requires boto library, version %s or later, (http://code.google.com/p/boto/).) is_secures3)hostportrcalling_formatzIt seems we previously failed to detect support for calling formats in the boto library, yet the support is there. This is almost certainly a duplicity bug.Boto requires a bucket name.)boto.s3.connectionr hasattrr r ImportErrorrs3_use_new_styler FatalError ErrorCodeboto_old_styleBOTO_MIN_VERSIONboto_lib_too_oldhostnameconnects3_unencrypted_connectionrboto_calling_formatrr) scheme parsed_url storage_urir r r cfs_supportedrconnr'A/usr/lib/python3/dist-packages/duplicity/backends/_boto_single.pyget_connection&sX          r)c@szeZdZdZddZddZddZdd Zd d Zd d Z ddZ ddZ ddZ ddZ ddZdddZddZdS) BotoBackenda Backend for Amazon's Simple Storage System, (aka Amazon S3), though the use of the boto module, (http://code.google.com/p/boto/). To make use of this backend you must set aws_access_key_id and aws_secret_access_key in your ~/.boto or /etc/boto.cfg with your Amazon Web Services key id and secret respectively. Alternatively you can export the environment variables AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY. cCstjj||z ddl}ddlm}Wntyw|jt ks$Jdd|j dD|_ |j r;|j d|_ntd|j|_|j rPdd|j |_nd|_tj||_||_d |jdd |j df|_tjrv|j|_nd|_|i|_dS) Nr)LocationcSsg|]}|dkr|qS)r').0xr'r'r( sz(BotoBackend.__init__../rz%s/r,z://) duplicitybackendBackend__init__botorr+rVersionrpathsplit url_partspop bucket_namerr"join key_prefixstrip_auth_from_url straight_urlr#lstrip boto_uri_strrs3_european_bucketsEU my_locationresetConnection _listed_keys)selfr#r6r+r'r'r(r5s4    zBotoBackend.__init__cCs(|`i|_d|_d|_d|_|`|`dSN)rGbucketr&r$rHr'r'r(_closeszBotoBackend._closecCsddl}t|ddr|jd|_d|_d|_|`|`||j|_t|j|j |j|_|j |j sA|jj |j |j d|_dS|j|j |_dS)Nrr&Flocation)r6getattrr&closerJr$rBr)r"r#r r< create_bucketrE get_bucket)rHr6r'r'r(rFs    zBotoBackend.resetConnectioncCs |dSrI)rFrKr'r'r(_retry_cleanups zBotoBackend._retry_cleanupc Cst|}tjrtjstdtjjd|j durHz |j |j |_ Wn%t yG}zdt|vr<|j j|j |jd|_ nWYd}~nd}~ww|j |j|}tjrWd}ntjr]d}ntjrcd}n tjrmd|vrmd }nd }td |j||ftjrd |d d}n(tjrtjdurtdtjjdd |dtjd}tjdurtj|d<nd |d}t}||j ||t}t!||pd} t"j#$|j | } t%d|j||| fdS)Nz`European bucket creation was requested, but not new-style bucket addressing (--s3-use-new-style))code NoSuchBucketrMREDUCED_REDUNDANCY STANDARD_IA ONEZONE_IAmanifestGLACIERSTANDARDzUploading %s/%s to %s Storagezapplication/octet-streamAES256) Content-Typex-amz-storage-classx-amz-server-side-encryptionzOS3 USE SSE KMS was requested, but key id not provided require (--s3-kms-key-id)zaws:kms)r]r^r_z+x-amz-server-side-encryption-aws-kms-key-idzx-amz-grant-full-control)r]r^z7Uploaded %s/%s to %s Storage at roughly %f bytes/second)&r fsdecoderrCrrrrs3_bucket_not_stylerJr&rRr< ExceptionrrQrEnew_keyr> s3_use_rrs s3_use_ias3_use_onezone_ias3_use_glacierInfor@ s3_use_sses3_use_sse_kms s3_kms_key_id s3_kms_no_id s3_kms_granttimeuploadnameabsosr8getsizeDebug) rH source_pathremote_filenameekey storage_classheaders upload_start upload_endtotal_srough_upload_speedr'r'r(_putsx          zBotoBackend._putcCsDt|}|j|}|j|dd|j|}|||jdS)NT)wait)r rar>pre_process_downloadrGrFget_contents_to_filenamerq)rHrw local_pathkey_nameryr'r'r(_gets   zBotoBackend._getcCs|jstd|S)NzNo connection to backend)rJrlist_filenames_in_bucketrKr'r'r(_listszBotoBackend._listc Cspg}|jj|jdD]+}z |j|jdd}||||j|j<td|j |fWq t y5Yq w|S)N)prefixr,r`z Listed %s/%s) rJlistr>ryreplaceappendrGrrur@AttributeError)rH filename_listkfilenamer'r'r(rs    z$BotoBackend.list_filenames_in_bucketcCs t|}|j|j|dSrI)r rarJ delete_keyr>)rHrr'r'r(_delete3s zBotoBackend._deletecCs6t|}|j|j|}|durddiSd|jiS)Nsize)r rarJr r>r)rHrryr'r'r(_query7s  zBotoBackend._querycCs0|j||tjtddtjdd|dS)Nr1i)cbnum_cb)set_contents_from_filenamer report_transfermaxrvolsizerP)rHrryr{r'r'r(rp>s  zBotoBackend.uploadFcCs|j|}|j|dst|j|d|j|<|j|}|jdkrc|j|jjs8t d||j dd|ret d||j|jjrZt d||j|jjsIt d |dSdSdS) NFrrZz.File %s is in Glacier storage, restoring to S3r`)daysz+Waiting for file %s to restore from Glacier<z.File %s was successfully restored from Glacier)r>rGgetrrJrzget_keyryongoing_restorerrirestorerosleeprF)rHrwrrryr'r'r(rEs"     z BotoBackend.pre_process_downloadcCsftdtdd}|D]}t|}||j|dq Wdn1s'wYtddS)Nz&Starting batch unfreezing from Glacierzs3-unfreeze-glacier)thread_name_prefixFz&Batch unfreezing from Glacier finished)rrirr rasubmitr)rHremote_filenamesexecutorrwr'r'r(pre_process_download_batchXs   z&BotoBackend.pre_process_download_batchN)F)__name__ __module__ __qualname____doc__r5rLrFrSrrrrrrrprrr'r'r'r(r*ys , B  r*) __future__rbuiltinsrconcurrent.futuresrrsroduplicity.backendr2rrduplicity.errorsrrr r rr)r3r4r*r'r'r'r(s       S