R j? BSrSSKrSSKrSSKrSSKrSSKrSSKrSSKrSSK r SSK r SSK J r SSKJr SSKJr SSKJrJrJrJrJrJrJrJrJrJrJr SSKJr SSK J!r! SS K"J#r#J$r$ SS K%J&r& SS K'J(r( SS K)J*r* SS K+J,r, SSK-J.r. SSK/J0r0 SSK1J2r2 SSK3J4r4 SSK5J6r6J7r7J8r8 \Rr"\:5r;\\<\<4r=S\"SS\?5r@S\!SS4SjrA"SS\?5rBS\zO The main purpose of this module is to expose LinkCollector.collect_sources(). N) dataclass) HTMLParser)Values) CallableDictIterableListMutableMapping NamedTupleOptionalProtocolSequenceTupleUnion)requests)Response) RetryErrorSSLError)NetworkConnectionError)Link) SearchScope) PipSession)raise_for_status)is_archive_fileredact_auth_from_url)vcs)CandidatesFromPage LinkSource build_sourceurlreturnc[RH?nUR5RU5(dM)U[ U5S;dM=Us $ g)z_Look for VCS schemes in the URL. Returns the matched VCS scheme, or None if there's no match. z+:N)rschemeslower startswithlen)r"schemes ID:\diplom_project\venv\Lib\site-packages\pip/_internal/index/collector.py_match_vcs_schemer+3sC ++ 99; ! !& ) )c#f+.>$.FM c8^\rSrSrS\S\SS4U4SjjrSrU=r$)_NotAPIContent> content_type request_descr#Nc<>[TU]X5 XlX lgN)super__init__r0r1)selfr0r1 __class__s r*r5_NotAPIContent.__init__?s 4((r,)r0r1)__name__ __module__ __qualname____firstlineno__strr5__static_attributes__ __classcell__r7s@r*r.r.>s")S))))r,r.responsecURRSS5nUR5nURS5(ag[ XR R 5e)z Check the Content-Type header to ensure the response contains a Simple API Response. Raises `_NotAPIContent` if the content type is not a valid content-type. Content-TypeUnknown)z text/htmlz#application/vnd.pypi.simple.v1+html#application/vnd.pypi.simple.v1+jsonN)headersgetr&r'r.requestmethod)rAr0content_type_ls r*_ensure_api_headerrKEs^##'' BL!'')N     '7'7'>'> ??r,c\rSrSrSrg)_NotHTTP[N)r9r:r;r<r>rOr,r*rMrM[sr,rMsessionc[RRU5up#pEnUS;a [5eUR USS9n[ U5 [ U5 g)z Send a HEAD request to the URL, and ensure the response contains a simple API Response. Raises `_NotHTTP` if the URL is not available for a HEAD request, or `_NotAPIContent` if the content type is not a valid content type. >httphttpsT)allow_redirectsN)urllibparseurlsplitrMheadrrK)r"rPr)netlocpathqueryfragmentresps r*_ensure_api_responser^_sR-3LL,A,A#,F)FD &&j <<T< 2DTtr,c[[U5R5(a [XS9 [R S[ U55 URUSR/SQ5SS.S9n[U5 [U5 [R S[ U5URRS S 55 U$) a5Access an Simple API response with GET, and return the response. This consists of three parts: 1. If the URL looks suspiciously like an archive, send a HEAD first to check the Content-Type is HTML or Simple API, to avoid downloading a large file. Raise `_NotHTTP` if the content type cannot be determined, or `_NotAPIContent` if it is not HTML or a Simple API. 2. Actually perform the request. Raise HTTP exceptions on network failures. 3. Check the Content-Type header to make sure we got a Simple API response, and raise `_NotAPIContent` otherwise. rPzGetting page %sz, )rEz*application/vnd.pypi.simple.v1+html; q=0.1ztext/html; q=0.01z max-age=0)Acceptz Cache-Control)rFzFetched page %s as %srCrD) rrfilenamer^loggerdebugrrGjoinrrKrF)r"rPr]s r*_get_simple_responserfqstCy))**S2 LL"$8$=> ;; ii()+   D4Tt LLS! 3 Kr,rFcU(aOSU;aI[RR5nUSUS'URS5nU(a [ U5$g)z=Determine if we have any encoding information in our headers.rCz content-typecharsetN)emailmessageMessage get_paramr=)rFmrhs r*_get_encoding_from_headersrnsK>W, MM ! ! ##N3.++i( w<  r,c>\rSrSrS SjrS\S\4SjrS\4Sjr Sr g) CacheablePageContentr#Nc6UR(deXlgr3)cache_link_parsingpager6rts r*r5CacheablePageContent.__init__s&&&& r,otherc[U[U55=(a- URRURR:H$r3) isinstancetypertr")r6rws r*__eq__CacheablePageContent.__eq__s-%d,P%**..1PPr,c@[URR5$r3)hashrtr"r6s r*__hash__CacheablePageContent.__hash__sDIIMM""r,)rt)rt IndexContentr#N) r9r:r;r<r5objectboolr{intrr>rOr,r*rprps)QFQtQ###r,rpc,\rSrSrSSS\\4SjrSrg) ParseLinksrtrr#cgr3rOrus r*__call__ParseLinks.__call__sr,rON)r9r:r;r<rrrr>rOr,r*rrsC^CCr,rfnc^^[R"SS9S[S[[4U4Sjj5m[R "T5SSS[[4UU4Sjj5nU$) z Given a function that parses an Iterable[Link] from an IndexContent, cache the function's result (keyed by CacheablePageContent), unless the IndexContent `page` has `page.cache_link_parsing == False`. N)maxsizecacheable_pager#c:>[T"UR55$r3)listrt)rrs r*wrapper*with_cached_index_content..wrappersB~**+,,r,rtrcj>UR(aT"[U55$[T"U55$r3)rsrpr)rtrrs r*wrapper_wrapper2with_cached_index_content..wrapper_wrappers+  " "/56 6BtH~r,) functools lru_cacherpr rwraps)rrrs` @r*with_cached_index_contentrsl&- 4-d-'-__Rnd r,rtrc#~# URR5nURS5(ac[R"UR 5nUR S/5H,n[R"X0R5nUcM(Uv M. g[UR5nUR=(d SnURUR RU55 URnUR=(d UnURH!n [R "XUS9nUcMUv M# g7f)zT Parse a Simple API's Index Content, and yield its anchor elements as Link objects. rEfilesNzutf-8)page_urlbase_url)r0r&r'jsonloadscontentrGr from_jsonr"HTMLLinkParserencodingfeeddecoderanchors from_element) rtrJdatafilelinkparserrr"ranchors r* parse_linksrs &&,,.N  !FGGzz$,,'HHWb)D>>$1D|J *  DHH %F}}'H KK ##H-. ((C%#H..  I <  !sD;D=T)frozencd\rSrSr%Sr\\S'\\S'\\\S'\\S'Sr \ \S'S \4S jr S r g ) ratRepresents one response (or page), along with its URL. :param encoding: the encoding to decode the given content. :param url: the URL from which the HTML was downloaded. :param cache_link_parsing: whether links parsed from this page's url should be cached. PyPI index urls should have this set to False, for example. rr0rr"Trsr#c,[UR5$r3)rr"rs r*__str__IndexContent.__str__ s#DHH--r,rON) r9r:r;r<__doc__bytes__annotations__r=r rsrrr>rOr,r*rrs:Nsm H##..r,c^\rSrSrSrS\SS4U4SjjrS\S\\\\ \4SS4S jr S\\\\ \4S\ \4S jr S r U=r $) rizZ HTMLParser that keeps the first base HREF and a list of all anchor elements' attributes. r"r#NcH>[TU]SS9 XlSUl/Ulg)NT)convert_charrefs)r4r5r"rr)r6r"r7s r*r5HTMLLinkParser.__init__s& $/'+ 79 r,tagattrscUS:Xa)URcURU5nUbX0lggUS:Xa%URR[ U55 gg)Nbasea)rget_hrefrappenddict)r6rrhrefs r*handle_starttagHTMLLinkParser.handle_starttagsR &=T]]2=='D $  CZ LL  U ,r,c.UHup#US:XdM Us $ g)NrrO)r6rnamevalues r*rHTMLLinkParser.get_href&s KDv~ !r,)rrr")r9r:r;r<rr=r5r rr rrr>r?r@s@r*rrsx :C:D:-3-tE#x}:L4M/N-SW-d5hsm);#<=(3-r,rrreasonmeth).Nc<Uc[RnU"SX5 g)Nz%Could not fetch URL %s: %s - skipping)rcrd)rrrs r*_handle_get_simple_failr-s  ||| 0$?r,rsc[UR5n[URURSUURUS9$)NrC)rr"rs)rnrFrrr")rArsrs r*_make_index_contentr7sE*(*:*:;H ( LL-  r,cURRSS5Sn[U5nU(a[R SUU5 g[ R RU5upEn nUS:Xa[RR[ RRU55(aQURS5(dUS- n[ R RUS5n[RSU5 [!X!S 9n[#XpR$S 9$![&a [R S U5 g[(a6n[R S UUR*UR,5 SnAgSnAf[.an[1X5 SnAgSnAf[2an[1X5 SnAgSnAf[4a2nS n U [7U5- n [1X [R8S9 SnAgSnAf[:R<an[1USU35 SnAgSnAf[:R>a [1US5 gf=f)N#rrzICannot look at %s URL %s because it does not support lookup as web pages.r/z index.htmlz# file: URL is directory, getting %sr`)rsz`Skipping page %s because it looks like an archive, and cannot be checked by a HTTP HEAD request.zSkipping page %s because the %s request got Content-Type: %s. The only supported Content-Types are application/vnd.pypi.simple.v1+json, application/vnd.pypi.simple.v1+html, and text/htmlz4There was a problem confirming the ssl certificate: )rzconnection error: z timed out) r"splitr+rcwarningrUrVurlparseosrZisdirrH url2pathnameendswithurljoinrdrfrrsrMr.r1r0rrrrr=inforConnectionErrorTimeout) rrPr" vcs_schemer)_rZr]excrs r*_get_index_contentrDs  ((..a  #C#3'J W    &||44S9FtQ1 BGGMM&..*E*Ed*KLL||C  3JC ll""3 5 :C@U#C9:#4rOr,r*rrs"*-..*-..r,rc \rSrSrSrS\S\SS4Sjr\SS\S\ S \ SS4S jj5r \ S\ \4S j5rS \S\\4S jrS\S\S\4SjrSrg) LinkCollectoriz Responsible for collecting Link objects from all configured locations, making network requests as needed. The class's main method is its collect_sources() method. rP search_scoper#NcX lXlgr3rrP)r6rPrs r*r5LinkCollector.__init__s ) r,optionssuppress_no_indexcDUR/UR-nUR(a5U(d.[R SSR SU555 /nUR =(d /n[R"UUURS9n[UUS9nU$)z :param session: The Session to use to make requests. :param suppress_no_index: Whether to ignore the --no-index option when constructing the SearchScope object. zIgnoring indexes: %s,c38# UHn[U5v M g7fr3r).0r"s r* 'LinkCollector.create..sIjs-c22js)rrno_index)rPr) index_urlextra_index_urlsrrcrdrerrcreater)clsrPrrrrrlink_collectors r*rLinkCollector.creates''(7+C+CC   $5 LL&IjII J''-2 "))!!%% '% r,c.URR$r3)rrrs r*rLinkCollector.find_linkss  +++r,locationc([XRS9$)z. Fetch an HTML page containing package links. r`)rrP)r6rs r*fetch_responseLinkCollector.fetch_responses"(LLAAr, project_namecandidates_from_pagec^^^[R"UUU4SjTRRT555R 5n[R"UUU4SjTR 55R 5n[ R[R5(a[R"XC5Vs/sH&nUcMURcMSUR3PM( nn[U5STS3/U-n[ RSRU55 [![#U5[#U5S9$s snf)Nc 3h># UH'n[UTTRRSSTS9v M) g7f)Frpage_validator expand_dirrsrNr!rPis_secure_originrlocrrr6s r*r0LinkCollector.collect_sources..s? 4 P %9#||<< #()  P/2c 3h># UH'n[UTTRRSSTS9v M) g7f)TrNr r s r*rr s= 5 ' %9#||<<#')  'rz* z' location(s) to search for versions of : )rr) collections OrderedDictrget_index_urls_locationsvaluesrrc isEnabledForloggingDEBUG itertoolschainrr(rdrerr)r6rrindex_url_sourcesfind_links_sourcessliness``` r*collect_sourcesLinkCollector.collect_sourcess8 (33 4 ((AA,O 4   &( )44 5  5   &(    w}} - -#);OOA%&VV"QVVH O  u:,##/.3E LL5) *./-.  sE  E !E r)F)r9r:r;r<rrrr5 classmethodrrrpropertyr r=rrr rrrrrr>rOr,r*rrs"   #(    B,DI,,BtB0FB , , 1,   , r,rr3)T)Rrr email.messagerirrrrr urllib.parserUurllib.request dataclassesr html.parserroptparsertypingrrrr r r r r rrr pip._vendorrpip._vendor.requestsrpip._vendor.requests.exceptionsrrpip._internal.exceptionsrpip._internal.models.linkr!pip._internal.models.search_scoperpip._internal.network.sessionrpip._internal.network.utilsrpip._internal.utils.filetypesrpip._internal.utils.miscrpip._internal.vcsrsourcesrr r! getLoggerr9rcr=ResponseHeadersr+ Exceptionr.rKrMr^rfrnrprrrrrrrrrrrrOr,r*r9s7  !"    !)@;*94899!AA   8 $ c*38C=)Y)@@d@, y cJ4$<c<J<8<~HSM # #DD*(n$8 $...(Z>+/@ @ #y. !@ 8I& '@ @48  ,0  :T:z:h~>V:z/z/ h h r,