o ~`q@sdZddlmZddlZddlZddlZddlZddlZddl Z ddl Z ddl Z ddl Z ddl Z ddlZddlmZddlmZddlmZmZmZdd lmZmZmZmZmZmZzdd l mZWneysdZdd l m!Z!Ynwd d gZ"dZ#ej#Z$ej%j&Z'e (Z)e *Z+dZ,ej-ej.ej/ej0ej1ej2ej3ej4ej5ej6ej7ej8ej9ej:ej;ejej?ej@ejAejBejCejDejEejFejGejHejIejJgZKejLejMejNfeejMejNfiZOePe drejQejQfeOe jR<ePe drejSejSfeOe jT<ePe drejMejMfeOe jU<ePe drejVejVfeOe jW<ePe dr!ejNejNfeOe jX<dd ZYdd ZZddZ[ddZ\e]e[Z^e_e\Z`GdddeaZberLd$ddZcnd%d!dZceceb_cGd"d#d#eaZddS)&a SecureTranport support for urllib3 via ctypes. This makes platform-native TLS available to urllib3 users on macOS without the use of a compiler. This is an important feature because the Python Package Index is moving to become a TLSv1.2-or-higher server, and the default OpenSSL that ships with macOS is not capable of doing TLSv1.2. The only way to resolve this is to give macOS users an alternative solution to the problem, and that solution is to use SecureTransport. We use ctypes here because this solution must not require a compiler. That's because pip is not allowed to require a compiler either. This is not intended to be a seriously long-term solution to this problem. The hope is that PEP 543 will eventually solve this issue for us, at which point we can retire this contrib module. But in the short term, we need to solve the impending tire fire that is Python on Mac without this kind of contrib module. So...here we are. To use this module, simply import and inject it:: import urllib3.contrib.securetransport urllib3.contrib.securetransport.inject_into_urllib3() Happy TLSing! This code is a bastardised version of the code found in Will Bond's oscrypto library. An enormous debt is owed to him for blazing this trail for us. For that reason, this code should be considered to be covered both by urllib3's license and by oscrypto's: .. code-block:: Copyright (c) 2015-2016 Will Bond Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. )absolute_importN)util)PROTOCOL_TLS_CLIENT)CoreFoundationSecurity SecurityConst)_assert_no_error_build_tls_unknown_ca_alert_cert_array_from_pem_create_cfstring_array_load_client_cert_chain_temporary_keychain) _fileobjectbackport_makefileinject_into_urllib3extract_from_urllib3Ti@PROTOCOL_SSLv2PROTOCOL_SSLv3PROTOCOL_TLSv1PROTOCOL_TLSv1_1PROTOCOL_TLSv1_2cCs.tt_ttj_tt_ttj_dt_dtj_dS)zG Monkey-patch urllib3 with SecureTransport-backed SSL-support. TN)SecureTransportContextr SSLContextssl_HAS_SNIIS_SECURETRANSPORTrrA/usr/lib/python3/dist-packages/urllib3/contrib/securetransport.pyr  cCs.tt_ttj_tt_ttj_dt_dtj_dS)z> Undo monkey-patching by :func:`inject_into_urllib3`. FN)orig_util_SSLContextrrrorig_util_HAS_SNIrrrrrr rr!c Csd}zt|}|durtjWS|j}|d}|}d}d}zA||kr_|dus,|dkr9t||s9tt j d||} t j |  ||} || | } || 7}| s[|sZtjWWSn||ks$Wn8tjy} z+| j }|dur|t j kr||d<|t jks|t jkrtjWYd} ~ WSWYd} ~ nd} ~ ww||d<||krtjWSWdSty} z|dur| |_tjWYd} ~ Sd} ~ ww)zs SecureTransport read callback. This is called by ST to request that data be returned from the socket. Nr timed out)_connection_refsgetr errSSLInternalsocket gettimeoutr wait_for_readerrorerrnoEAGAINctypesc_char from_address recv_intoerrSSLClosedGraceful ECONNRESETEPIPEerrSSLClosedAborterrSSLWouldBlock Exception _exception) connection_id data_bufferdata_length_pointerwrapped_socket base_socketrequested_lengthtimeoutr+ read_count remainingbuffer chunk_sizeerrr _read_callbacks\      rEc Csnd}zt|}|durtjWS|j}|d}t||}|}d}d} z.| |krR|dus2|dkr?t ||s?t t j d| |} | | 7} || d}| |ks*Wn8tj y} z+| j }|dur|t j kr| |d<|t jksv|t jkrtjWYd} ~ WSWYd} ~ nd} ~ ww| |d<| |krtjWSWdSty} z|dur| |_tjWYd} ~ Sd} ~ ww)zx SecureTransport write callback. This is called by ST to request that data actually be sent on the network. Nrr$)r%r&r r'r(r. string_atr)rwait_for_writer+r,r-sendr3r4r5r6r7r8) r9r:r;r<r=bytes_to_writedatar?r+sent chunk_sentrDrrr _write_callback sP       rMc@seZdZdZddZejddZddZdd Z d d Z d d Z ddZ ddZ ddZddZd.ddZddZddZddZdd Zd!d"Zd#d$Zd/d&d'Zd(d)Zd*d+Zd,d-ZdS)0 WrappedSocketz API-compatibility wrapper for Python's OpenSSL wrapped socket object. Note: _makefile_refs, _drop(), and _reuse() are needed for the garbage collector of PyPy. cCsL||_d|_d|_d|_d|_d|_d|_d|_|j|_ |j ddSNrF) r(context_makefile_refs_closedr8 _keychain _keychain_dir_client_cert_chainr)_timeout settimeout)selfr(rrr __init__Ns zWrappedSocket.__init__ccs6d|_dV|jdur|jd}|_||dS)a] A context manager that can be used to wrap calls that do I/O from SecureTransport. If any of the I/O callbacks hit an exception, this context manager will correctly propagate the exception after the fact. This avoids silently swallowing those exceptions. It also correctly forces the socket closed. N)r8close)rX exceptionrrr _raise_on_error`s  zWrappedSocket._raise_on_errorcCs2tjttt}t|j|tt}t|dS)a4 Sets up the allowed ciphers. By default this matches the set in util.ssl_.DEFAULT_CIPHERS, at least as supported by macOS. This is done custom and doesn't allow changing at this time, mostly because parsing OpenSSL cipher strings is going to be a freaking nightmare. N)rSSLCipherSuitelen CIPHER_SUITESSSLSetEnabledCiphersrPr )rXciphersresultrrr _set_ciphersus   zWrappedSocket._set_ciphersc CsD|sdSt|}zt|j|}t|Wt|dSt|w)z< Sets up the ALPN protocols on the context. N)r rSSLSetALPNProtocolsrPr r CFRelease)rX protocols protocols_arrrbrrr _set_alpn_protocolss z!WrappedSocket._set_alpn_protocolsc Cs|sdStjtjf}z||}||vrWdSd|f}Wnty4}z d|f}WYd}~nd}~wwt|}|j|t ddd}|j tj tj ||td|)z Called when we have set custom validation. We do this in two cases: first, when cert validation is entirely disabled; and second, when using a custom trust DB. Raises an SSLError if the connection is not trusted. Nzerror code: %dz exception: %riirrzcertificate verify failed, %s)r kSecTrustResultUnspecifiedkSecTrustResultProceed_evaluate_trustr7r versionr(sendallstructpack setsockopt SOL_SOCKET SO_LINGERrZsslSSLError) rXverify trust_bundle successes trust_resultreasonrDrecoptsrrr _custom_validates(   zWrappedSocket._custom_validatec Cstj|rt|d }|}Wdn1swYd}t}zRt|}t|j t |}t ||s?t dt||}t |t|d}t |t}t|t |}t |W|rlt||durut||jS|rt||durt|ww)NrbzFailed to copy trust referenceT)ospathisfileopenreadr SecTrustRefr SSLCopyPeerTrustrPr.byrefr rtruSecTrustSetAnchorCertificates!SecTrustSetAnchorCertificatesOnlySecTrustResultTypeSecTrustEvaluaterrevalue)rXrwf cert_arraytrustrbryrrr rls:           zWrappedSocket._evaluate_trustc Cstdtjtj|_t|jtt} t | t t |d} | t vr-| dd} | t vs#|t | <Wdn1s;wYt |j| } t | |ret|tsW|d}t|j|t|} t | ||| t|j|} t | t|j|} t | |r|durt|jtjd} t | |rt\|_|_t|j|||_t|j|j} t |  |1t |j} | tj!krt"#d| tj$kr|%|| Wdqt |  WddS1swYq)z Actually performs the TLS handshake. This is run automatically by wrapped socket, and shouldn't be needed in user code. Nirzutf-8Tzhandshake timed out)&rSSLCreateContextr kSSLClientSidekSSLStreamTyperP SSLSetIOFuncs_read_callback_pointer_write_callback_pointerr _connection_ref_lockidr%SSLSetConnection isinstancebytesencodeSSLSetPeerDomainNamer^rcrhSSLSetProtocolVersionMinSSLSetProtocolVersionMaxSSLSetSessionOption"kSSLSessionOptionBreakOnServerAuthrrSrTrrUSSLSetCertificater\ SSLHandshaker6r(r?errSSLServerAuthCompletedr}) rXserver_hostnamervrw min_version max_version client_cert client_keyclient_key_passphrasealpn_protocolsrbhandlerrr handshakesl                  zWrappedSocket.handshakecCs |jSN)r(filenorXrrr r7 zWrappedSocket.filenocCs.|jdkr |jd8_|jr|dSdS)Nrr)rQrRrZrrrr _decref_socketios;s  zWrappedSocket._decref_socketioscCs&t|}|||}|d|}|Sr)r.create_string_bufferr1)rXbufsizrB bytes_readrJrrr recvAs   zWrappedSocket.recvNc Cs|jrdS|dur t|}tj||}td}|t|j ||t |}Wdn1s5wY|t j krL|j dkrItd|j S|t jt jfvr[||j St||j S)Nrzrecv timed out)rRr^r.r/ from_bufferc_size_tr\rSSLReadrPrr r6rr(r?r2errSSLClosedNoNotifyrZr )rXrBnbytesprocessed_bytesrbrrr r1Gs.     zWrappedSocket.recv_intocC ||_dSrrV)rXr?rrr rWrrzWrappedSocket.settimeoutcC|jSrrrrrr r)uszWrappedSocket.gettimeoutc Csztd}|t|j|t|t|}Wdn1s"wY|tj kr6|j dkr6t dt ||j S)Nrzsend timed out)r.rr\rSSLWriterPr^rr r6rr(r?r )rXrJrrbrrr rHxs   zWrappedSocket.sendcCsBd}|t|kr||||t}||7}|t|ksdSdSNr)r^rHSSL_WRITE_BLOCKSIZE)rXrJ total_sentrKrrr rns  zWrappedSocket.sendallcCs:|t|jWddS1swYdSr)r\rSSLCloserPrrrr shutdowns "zWrappedSocket.shutdowncCs|jdkr@d|_|jrt|jd|_|jr t|jd|_|jr;t|jt|jt |j d|_|_ |j S|jd8_dS)NrT)rQrRrPrrerUrSrSecKeychainDeleteshutilrmtreerTr(rZrrrr rZs        zWrappedSocket.closeFc Cs |stdt}d}d}zqt|jt|}t||s2W|r't ||r0t |dSdSt |}|sLW|rAt ||rJt |dSdSt |d}|sVJt |}|s_Jt |}t|} t| |}W|rwt ||r~t ||S|rt ||rt |ww)Nz2SecureTransport only supports dumping binary certsr) ValueErrorrrrrPr.rr rreSecTrustGetCertificateCountSecTrustGetCertificateAtIndexSecCertificateCopyDataCFDataGetLengthCFDataGetBytePtrrF) rX binary_formrcertdata der_bytesrb cert_countleaf data_lengthr:rrr getpeercertsP           zWrappedSocket.getpeercertcCst}t|jt|}t||jtj krt d|jtj kr%dS|jtj kr-dS|jtjkr5dS|jtjkr=dS|jtjkrEdSt d|)Nz(SecureTransport does not support TLS 1.3zTLSv1.2zTLSv1.1TLSv1SSLv3SSLv2zUnknown TLS version: %r)r SSLProtocolSSLGetNegotiatedProtocolVersionrPr.rr rr kTLSProtocol13rtrukTLSProtocol12kTLSProtocol11 kTLSProtocol1 kSSLProtocol3 kSSLProtocol2)rXprotocolrbrrr rms$        zWrappedSocket.versioncCs|jd7_dSNr)rQrrrr _reuseszWrappedSocket._reusecCs(|jdkr |dS|jd8_dSr)rQrZrrrr _drops  zWrappedSocket._dropr)F)__name__ __module__ __qualname____doc__rY contextlibcontextmanagerr\rcrhr}rlrrrrr1rWr)rHrnrrZrrmrrrrrr rNFs0   #(] + : rNcCs|jd7_t|||ddS)NrT)rZ)rQr)rXmodebufsizerrr makefilesrrcOsd}t|||g|Ri|Srr)rXr bufferingargskwargsrrr rsc@seZdZdZddZeddZejddZeddZejd dZed d Z e jd d Z d dZ ddZ ddZ dddZ dddZddZ    d ddZdS)!rz I am a wrapper class for the SecureTransport library, to translate the interface of the standard library ``SSLContext`` object to calls into SecureTransport. cCs>t|\|_|_d|_d|_d|_d|_d|_d|_d|_ dSrO) _protocol_to_min_max _min_version _max_version_options_verify _trust_bundle _client_cert _client_key_client_key_passphrase_alpn_protocols)rXrrrr rYs zSecureTransportContext.__init__cCdS) SecureTransport cannot have its hostname checking disabled. For more, see the comment on getpeercert() in this file. Trrrrr check_hostname#z%SecureTransportContext.check_hostnamecCr)rNrrXrrrr r+rcCrrrrrrr options3szSecureTransportContext.optionscCrrrrrrr r=s cCs|jrtjStjSr)rrt CERT_REQUIRED CERT_NONErrrr verify_modeBsz"SecureTransportContext.verify_modecCs|tjkr d|_dSd|_dS)NTF)rtrrrrrr rFscCsdSrrrrrr set_default_verify_pathsJs z/SecureTransportContext.set_default_verify_pathscCs|Sr)rrrrr load_default_certsVsz)SecureTransportContext.load_default_certscCs|tjjkr tddS)Nz5SecureTransport doesn't support custom cipher strings)rrDEFAULT_CIPHERSr)rXrarrr set_ciphersYs z"SecureTransportContext.set_ciphersNcCsN|durtd|dur t|Wdn1swY|p#||_dS)Nz1SecureTransport does not support cert directories)rrr)rXcafilecapathcadatarrr load_verify_locations^s z,SecureTransportContext.load_verify_locationscCs||_||_||_dSr)rr_client_cert_passphrase)rXcertfilekeyfilepasswordrrr load_cert_chainjs z&SecureTransportContext.load_cert_chaincCs&ttds tddd|D|_dS)z Sets the ALPN protocols that will later be set on the context. Raises a NotImplementedError if ALPN is not supported. rdz2SecureTransport supports ALPN only in macOS 10.12+cSsg|]}t|qSr)six ensure_binary).0prrr ysz=SecureTransportContext.set_alpn_protocols..N)hasattrrNotImplementedErrorr)rXrfrrr set_alpn_protocolsos z)SecureTransportContext.set_alpn_protocolsFTc CsN|rJ|sJ|s Jt|}|||j|j|j|j|j|j|j|j |Sr) rNrrrrrrrrr)rXsock server_sidedo_handshake_on_connectsuppress_ragged_eofsrr<rrr wrap_socket{s   z"SecureTransportContext.wrap_socket)NNN)NN)FTTN)rrrrrYpropertyrsetterrrrrrr rrrrrrr rs4         r)r)rN)er __future__rrr.r,os.pathrrr(rtro threadingweakrefrr util.ssl_r_securetransport.bindingsrrr _securetransport.low_levelr r r r rrr ImportErrorpackages.backports.makefiler__all__rr#rrr"WeakValueDictionaryr%Lockrr'TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384'TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256%TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384%TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256-TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256+TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256#TLS_DHE_RSA_WITH_AES_256_GCM_SHA384#TLS_DHE_RSA_WITH_AES_128_GCM_SHA256'TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384$TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA'TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256$TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA%TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384"TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA%TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256"TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA#TLS_DHE_RSA_WITH_AES_256_CBC_SHA256 TLS_DHE_RSA_WITH_AES_256_CBC_SHA#TLS_DHE_RSA_WITH_AES_128_CBC_SHA256 TLS_DHE_RSA_WITH_AES_128_CBC_SHATLS_AES_256_GCM_SHA384TLS_AES_128_GCM_SHA256TLS_RSA_WITH_AES_256_GCM_SHA384TLS_RSA_WITH_AES_128_GCM_SHA256TLS_AES_128_CCM_8_SHA256TLS_AES_128_CCM_SHA256TLS_RSA_WITH_AES_256_CBC_SHA256TLS_RSA_WITH_AES_128_CBC_SHA256TLS_RSA_WITH_AES_256_CBC_SHATLS_RSA_WITH_AES_128_CBC_SHAr_ PROTOCOL_TLSrrrrrrrrrrrrrrrErM SSLReadFuncr SSLWriteFuncrobjectrNrrrrrr s 5     %         7 6 <