o U hh@sdZddlZddlmZmZddlmZddlmZm Z zddl m Z Wne y3ddlm Z YnwddlZddlZddlZddlZddlZddlmZmZmZdd lmZdd lmZdd lmZmZdd lmZdd lm Z m!Z!m"Z"m#Z#m$Z$m%Z%m&Z&zddl'Z'Wn e(ydZ)YnwdZ)ej*dkrdZ)ddl+m,Z,m-Z-ddl.m/Z/ddl0m1Z1ddl2m3Z3m4Z4m5Z5m6Z6m7Z7dZ8dZ9e8e9fZ:dZ;dZe?fe@e>fe@e>fZAe!e>ge=e?eAffZBe!e?eCgdfZDe=e?e#e>fZEGdddeFZGGddde$ZHGdddeHZIGd d!d!ZJ dd"e!e?ge>fd#e#e?d$e=e@e?e#e?ffd%d&ZKGd'd(d(eFZLGd)d*d*ZMd+ZNdeNfd,e!e?ge>fd-eMd.eOd/e?d$e>f d0d1ZPd2d3ZQd4d5ZRdd6d7ZSd8d9ZTd:d;ZUe=e>e?e#e?fZVGdd?d?eWZXGd@dAdAeWZYGdBdCdCeYZZGdDdEdEeYZ[d$e=e?e?ffdFdGZ\dHe&e>ee>fd$e?fdIdJZ]dddeNfdKe!e?ge>fd,e#e!e?ge>fd$e=eMe>ffdLdMZ^dNdOZ_GdPdQdQZ`GdRdSdSe`ZadTdUZbddWdXZcGdYdZdZZde%d[ZeGd\d]d]e"eeZfGd^d_d_efeMZgGd`dadaefeVZhGdbdcdcefe4ZiGdddedeZjGdfdgdgZkdhdiZlddkdlZmddmdnZndddjdodpe&ee4ee=e4e#e>ffdqe#eOdre#e?dse?fdtduZodvdwZpddxdyZqdddzd{eId|ere>d}e#ere>d$eeMfd~dZsddddpe&ee>ee=e4e#e>ffde#e?d$eeMfddZtdpe&ee4ee=e4e#eEffd$ee4fddZu dde#e?fddZvddddddpe&ee4ee=e4e#e>ffdqe#eOdre#e?deOd$e=e?eeMff ddZw      dd{eId|ee=e3e#eEfdre#e?dqe#eOdeOdeOd}e#ere>d$eeMfddZxde4d$eMfddZy    j dd{eId|ee=e3e#eEfdre#e?dqe#eOdeOdse?d}e#ere>fddZzdddjddpe&ee4ee=e4e#e>ffdre#e?dqe#eOdse?fddZ{GdddZ|dddjddeeMfddZ}ddZ~d$e>fddZdZddZddZddZdeeVde>d$e>fddZeZGdddZdjddde d|ere3d$e=e>e@ffddZz ddlmZmUZUWdSe(yYdSw)aClasses for dealing with packed git objects. A pack is a compact representation of a bunch of objects, stored using deltas where possible. They have two parts, the pack file, which stores the data, and an index that tells you where the data is. To find an object you look in all of the index files 'til you find a match for the object name. You then use the pointer got from this as a pointer in to the corresponding packfile. N) defaultdictdeque)suppress)BytesIOUnsupportedOperation)CSequenceMatcher)SequenceMatcher)IterableIteratorSequence)sha1)chain)SEEK_CURSEEK_END) unpack_from)BinaryIOCallableGenericOptionalProtocolTypeVarUnionFTPlan9)ApplyDeltaErrorChecksumMismatch)GitFile) LRUSizeCache)ObjectIDShaFile hex_to_sha object_header sha_to_hex ic@seZdZdZdddZdS)UnresolvedDeltasz$Delta objects could not be resolved.returnNcC ||_dSNshas)selfr+r-`/var/www/html/construction_image-detection-poc/venv/lib/python3.10/site-packages/dulwich/pack.py__init__g zUnresolvedDeltas.__init__r'N)__name__ __module__ __qualname____doc__r/r-r-r-r.r&dsr&c@szeZdZdeddfddZ ddeeeeefdee egdfddfdd Z d e de fd d Z d e defd dZdS)ObjectContainerobjr'NcCdS)z)Add a single object to this object store.Nr-r,r7r-r-r. add_objectlzObjectContainer.add_objectobjectsprogresscCr8)zAdd a set of objects to this object store. Args: objects: Iterable over a list of (object, path) tuples Nr-)r,r<r=r-r-r. add_objectsor;zObjectContainer.add_objectsr cCr8)zCheck if a hex sha is present.Nr-r,r r-r-r. __contains__zr;zObjectContainer.__contains__cCr8)zRetrieve an object.Nr-r?r-r-r. __getitem__}r;zObjectContainer.__getitem__r))r2r3r4rr:r tuplerstrrr>bytesboolr@rAr-r-r-r.r6ks  r6c @szeZdZdddededdfddZdd d eed edeefd d Z   dd e eded edededf ddZ dS)PackedObjectContainerF include_compr rHr'UnpackedObjectcC t|j)zGet a raw unresolved object.)NotImplementedErrorget_unpacked_object)r,r rHr-r-r.rLs z)PackedObjectContainer.get_unpacked_object allow_missingr+rNcCrJr))rKiterobjects_subsetr,r+rNr-r-r.rOs z(PackedObjectContainer.iterobjects_subsetTconvert_ofs_deltacCrJr))rKiter_unpacked_subset)r,r+rHrNrQr-r-r.rR z*PackedObjectContainer.iter_unpacked_subsetN)FFT) r2r3r4rDrErLr r rrOsetrRr-r-r-r.rFs>  rFc@s,eZdZdedfddZdefddZdS)UnpackedObjectStreamr'rIcCrJr))rK__iter__r,r-r-r.rVr0zUnpackedObjectStream.__iter__cCrJr)rK__len__rWr-r-r.rYr0zUnpackedObjectStream.__len__N)r2r3r4r rVintrYr-r-r-r.rUsrUreadcrc32r'cCsng}t|dks|dd@r3|d}|durt||}|t|ddt|dks|dd@s||fS)zVRead bytes marked with most significant bit. Args: read: Read function rrN)lenbinasciir\appendord)r[r\retbr-r-r.take_msb_bytess rec@seZdZdddZdS)PackFileDisappearedr'NcCr(r))r7r9r-r-r.r/r0zPackFileDisappeared.__init__r1)r2r3r4r/r-r-r-r.rfsrfc@seZdZUdZgdZeeed<eee ed<e de efed<ee ed<eee ed<ddddddd  dd d Z d dZ ddZ d efddZddZddZd efddZdS)rIaClass encapsulating an object unpacked from a pack file. These objects should only be created from within unpack_object. Most members start out as empty and are filled in at various points by read_zlib_chunks, unpack_object, DeltaChainIterator, etc. End users of this object should take care that the function they're getting this object from is guaranteed to set the members they need. ) _sha comp_chunksr\ decomp_chunks decomp_len delta_base obj_chunks obj_type_numoffset pack_type_numrmrlNrkrirh)rkrjr\sharirnr'cCs||_||_||_||_d|_|pg|_|dur%|dur%ttt||_ n||_ ||_ |t vr7d|_ d|_ dS||_ |j|_ ||_dSr))rnrgrorkrhrisummapr_rjr\ DELTA_TYPESrmrl)r,rorkrjr\rprirnr-r-r.r/s     zUnpackedObject.__init__cCs |jdur t|j|j|_|jS)z%Return the binary SHA of this object.N)rgobj_sharmrlrWr-r-r.rps zUnpackedObject.shacCs(|jdur |jdus Jt|j|jS)z"Return a ShaFile from this object.N)rmrlrfrom_raw_chunksrWr-r-r.sha_fileszUnpackedObject.sha_filecCs0|jtvrt|jttfsJ|j|jfS|jS)z>Return the decompressed chunks, or (delta base, delta chunks).)rors isinstancerkrDrZrirWr-r-r._objs  zUnpackedObject._objcCs8t|tsdS|jD]}t||t||krdSq dSNFT)rwrI __slots__getattr)r,otherslotr-r-r.__eq__ s  zUnpackedObject.__eq__cCs ||k Sr)r-r,r|r-r-r.__ne__r0zUnpackedObject.__ne__cs*fddjD}djjd|S)Ncs g|] }|dt|qS)=)r{.0srWr-r. s z+UnpackedObject.__repr__..z{}({})z, )rzformat __class__r2joinr,datar-rWr.__repr__szUnpackedObject.__repr__r1)r2r3r4r5rzrrZ__annotations__listrDrr/rprvOldUnpackedObjectrxr~rrCrr-r-r-r.rIs.    rIi read_someunpackedrH buffer_sizec Cs|jdkr tdt}g}|j}d}|j} ||} | s#td|| || } |t | 7}|| |j } | r\t | } |durPt | d| |}|r[| d| |d<n |durft | |}q|duro|dM}||jkrytd||_|r||_ | S) aRead zlib data from a buffer. This function requires that the buffer have additional data following the compressed data, which is guaranteed to be the case for git pack files. Args: read_some: Read function that returns at least one byte, but may return less than the requested size. unpacked: An UnpackedObject to write result data to. If its crc32 attr is not None, the CRC32 of the compressed bytes will be computed using this starting CRC32. After this function, will have the following attrs set: * comp_chunks (if include_comp is True) * decomp_chunks * decomp_len * crc32 include_comp: If True, include compressed data in the result. buffer_size: Size of the read buffer. Returns: Leftover unused data from the decompression. Raises: zlib.error: if a decompression error occurred. r]z+non-negative zlib data stream size expectedrTzEOF before end of zlib streamNz.decompressed data does not match expected size) rj ValueErrorzlib decompressobjrir\errorra decompressr_ unused_datar`rh) rrrHr decomp_objrhrirjr\adddecompunusedleftr-r-r.read_zlib_chunkssD         rcCs(t}|D]}||q|dS)zReturn the hexdigest of the SHA1 over a set of names. Args: iter: Iterator over string objects Returns: 40-byte hex sha1 digest ascii)r update hexdigestencode)iterrpnamer-r-r. iter_sha1`s rcCs8t|d }t||WdS1swYdS)zLoad an index file by path. Args: path: Path to the index file Returns: A PackIndex loaded from the given path rbN)rload_pack_index_file)pathfr-r-r.load_pack_indexms $rc Csz|}Wn ttfyd}Ynw|dur<|dur"t|j}trLr)filerrzUnknown pack index format )rstructunpack PackIndex2KeyError PackIndex1)rrrrversionr-r-r.rs rcCs\||ksJ||kr,||d}||}||kr|d}n ||kr&|d}n|S||ks dS)aFind a SHA in a data blob with sorted SHAs. Args: start: Start index of range to search end: End index of range to search sha: Sha to find unpack_name: Callback to retrieve SHA by index Returns: Index of the SHA, or None if it wasn't found rrNr-)startendrp unpack_nameifile_shar-r-r.bisect_find_shas     rc@seZdZdZddZddZdefddZdee fd d Z dee fd d Z de fd dZ de defddZde defddZdede fddZde defddZde fddZdee fddZd"ddZd"d d!ZdS)# PackIndexzAn index in to a packfile. Given a sha id of an object a pack index can tell you the location in the packfile of that object if it has it. cCsHt|tsdSt||D]\\}}}\}}}||kr!dSqdSry)rwrzip iterentries)r,r|name1_name2r-r-r.r~s  zPackIndex.__eq__cCs || Sr))r~rr-r-r.r zPackIndex.__ne__r'cCrJ)0Return the number of entries in this pack index.rXrWr-r-r.rY zPackIndex.__len__cCstt|S)z#Iterate over the SHAs in this pack.)rrr"_itersharWr-r-r.rVzPackIndex.__iter__cCrJ)Iterate over the entries in this pack index. Returns: iterator over tuples with object name, offset in packfile and crc32 checksum. )rKrrWr-r-r.r zPackIndex.iterentriescCrJ)pReturn the SHA1 checksum stored for the corresponding packfile. Returns: 20-byte binary digest )rKget_pack_checksumrWr-r-r.rs zPackIndex.get_pack_checksumrpcCstjdtdd||S)Nz Please use object_offset insteadr stacklevel)warningswarnDeprecationWarning object_offsetr,rpr-r-r. object_indexs zPackIndex.object_indexcCrJ)Return the offset in to the corresponding packfile for the object. Given the name of an object it will return the offset that object lives at within the corresponding pack file. If the pack file doesn't have the object then None will be returned. )rKrrr-r-r.rrSzPackIndex.object_offsetindexcCs,|D] \}}}||kr|Sqt|)zzMemoryPackIndex.for_packcCs|||Sr))rr)r other_indexr-r-r.cloneBrzMemoryPackIndex.cloner)r1)r2r3r4r5r/rrZrYrrrr classmethodrrr-r-r-r.rs  rcsDeZdZUdZeeed<d/d0ddZede fddZ fd d Z d0d d Z defd dZ dedefddZdefddZdefddZdeefddZdeefddZdeefddZdefddZd0dd Zdefd!d"Zdefd#d$Zdefd%d&Zd'edefd(d)Zd'edefd*d+Zd,edeefd-d.ZZ S)1 FilePackIndexaPack index that is based on a file. To do the loop it opens the file, and indexes first 256 4 byte groups with the first byte of the sha id. The value in the four byte group indexed is the end of the group that shares the same starting byte. Subtract one from the starting byte and index again to find the start of the group. The values are sorted by sha id within the group, so do the math to find the start and end offset and then bisect in to find if the value is present. _fan_out_tableNr'cCsT||_|durt|d|_n||_|dur!t|j|\|_|_dS|||_|_dS)zCreate a pack index object. Provide it with the name of the index file to consider, and it will map it whenever required. Nr) _filenamer_filer _contents_sizer,filenamerrrr-r-r.r/UszFilePackIndex.__init__cCrr)rrWr-r-r.rgzFilePackIndex.pathcs&t|tr |j|jkr dSt|SNF)rwrrsuperr~rrr-r.r~ks   zFilePackIndex.__eq__cCs.|jt|jdddur|jdSdS)Nr)rrr{rrWr-r-r.rus zFilePackIndex.closecCs |jdS)rr])rrWr-r-r.rYzrzFilePackIndex.__len__rcCrJ)zUnpack the i-th entry in the index file. Returns: Tuple with object name (SHA), offset in pack file and CRC32 checksum (if known). )rK _unpack_entryr,rr-r-r.r~rzFilePackIndex._unpack_entrycCrJ)z)Unpack the i-th name from the index file.)rK _unpack_namerr-r-r.rrzFilePackIndex._unpack_namecCrJ)z2Unpack the i-th object offset from the index file.)rK_unpack_offsetrr-r-r.rrzFilePackIndex._unpack_offsetcCrJ)zAUnpack the crc32 checksum for the ith object from the index file.)rK_unpack_crc32_checksumrr-r-r.rrz$FilePackIndex._unpack_crc32_checksumccs$tt|D]}||VqdSr))ranger_rrr-r-r.rszFilePackIndex._itershaccs$tt|D]}||VqdS)rN)rr_rrr-r-r.rszFilePackIndex.iterentries start_offsetcCsNg}tdD]}|j||d||dd}|td|dq|S)Nrr>Lr)rrrarr)r,rrcr fanout_entryr-r-r._read_fan_out_tables z!FilePackIndex._read_fan_out_tablecC&|}|}||krt||dS)z;Check that the stored checksum matches the actual checksum.Nrget_stored_checksumrr,actualstoredr-r-r.r  zFilePackIndex.checkcCst|jddS)zlCalculate the SHA1 checksum over this pack index. Returns: This is a 20-byte binary digest N)r rdigestrWr-r-r.rsz FilePackIndex.calculate_checksumcCt|jddS)rir rDrrWr-r-r.rzFilePackIndex.get_pack_checksumcCr")z`Return the SHA1 checksum stored for this index. Returns: 20-byte binary digest r Nr#rWr-r-r.rr$z!FilePackIndex.get_stored_checksumrpc Cs\t|dkr t|}z||WSty-}zt|jdd}|dvr(t||d}~ww)rrclosedN)NT)r_r rrr{rrf)r,rpexcr%r-r-r.rs   zFilePackIndex.object_offsetcCspt|dksJt|dd}|dkrd}n|j|d}|j|}t||||j}|dur3t|||S)rNrr)r_rbrrrrr)r,rpidxrrrr-r-r.rs  zFilePackIndex._object_offsetprefixccst|dd}|dkrd}n|j|d}t|ddd}|dkr*t|}n|j|}||ks5Jd}t||D]}||}||rN|Vd}q<|rSdSqL20srr)r,rrnrr-r-r.rs zPackIndex1._unpack_entrycCs"d|dd}|j||dS)Nr1r2rr')rr,rrnr-r-r.r szPackIndex1._unpack_namecCsd|d}td|j|dS)Nr1r2rrr3r4r-r-r.rs zPackIndex1._unpack_offsetcCrr)r-rr-r-r.rsz!PackIndex1._unpack_crc32_checksumr-r1 r2r3r4r5rCr/rrrrr/r-r-r r.rsrcsLeZdZdZddeddffdd ZddZd d Zd d Zd dZ Z S)rzVersion 2 Pack Index file.Nrr'cst|||||jdddkrtdtd|jd\|_|jdkr,td|j|d|_d|_|jd t ||_ |j dt ||_ |j dt ||_ dS) NrrzNot a v2 pack index filerr Version was rir') r r/rAssertionErrorrrrr_name_table_offsetr__crc32_table_offset_pack_offset_table_offset_pack_offset_largetable_offsetrr r-r.r/s  zPackIndex2.__init__cCs||||||fSr))rrrrr-r-r.r)szPackIndex2._unpack_entrycCs |j|d}|j||dSNr')r8rr4r-r-r.r0szPackIndex2._unpack_namecCsP|j|d}td|j|d}|d@r&|j|d@d}td|j|d}|S)Nrrrirz>Q)r:rrr;r4r-r-r.r4s zPackIndex2._unpack_offsetcCstd|j|j|ddS)Nrrr)rrr9rr-r-r.r<sz!PackIndex2._unpack_crc32_checksumr-r5r-r-r r.rsrcCsl|d}|s td|dddkrtd|td|d\}|dvr+td |td|d \}||fS) zRead the header of a pack file. Args: read: Read function Returns: Tuple of (pack version, number of objects). If no data is available to read, returns (None, None). zfile too short to contain packNrPACKzInvalid pack header r)rr6r)r7r)r[headerr num_objectsr-r-r.read_pack_header@srCchunkscCs t|tr t|Sttt|Sr))rwrDr_rqrr)rDr-r-r. chunks_lengthTs rEread_allcCsR|dur|}|r d}nd}t||d\}}|dd?d@}|dd@}t|ddD]\} } || d@| dd>7}q+t|} |tkrzt||d\}}| t|7} |d d @rZt|dd@} |ddD]} | d7} | dK} | | d@7} qf| } n|tkr|d }|durt||}|} | d 7} nd} t|| ||d }t ||||d }||fS)aUnpack a Git object. Args: read_all: Read function that blocks until the number of requested bytes are read. read_some: Read function that returns at least one byte, but may not return the number of bytes requested. compute_crc32: If True, compute the CRC32 of the compressed data. If False, the returned CRC32 will be None. include_comp: If True, include compressed data in the result. zlib_bufsize: An optional buffer size for zlib operations. Returns: A tuple of (unpacked, unused), where unused is the unused data leftover from decompression, and unpacked in an UnpackedObject with the following attrs set: * obj_chunks (for non-delta types) * pack_type_num * delta_base (for delta types) * comp_chunks (if include_comp is True) * decomp_chunks * decomp_len * crc32 (if compute_crc32 is True) Nr)r\rr$rr]r^r')rkrjr\)rrH) re enumerater_ OFS_DELTAr7 REF_DELTAr`r\rIr)rFr compute_crc32rH zlib_bufsizer\rawtype_numrrbyteraw_basedelta_base_offsetrkdelta_base_objrrr-r-r. unpack_object[sN      rTcCs$|\}}|tvrt|dSt|S)zBCompute the size of a unresolved object for use with LRUSizeCache.r)rsrE)valuenumr7r-r-r._compute_object_sizes rWc@sneZdZdZdefdddZddZdd Zed d Z d d Z ddZ de fddZ ddeefddZdS)PackStreamReaderz|Class to read a pack stream. The pack is read from a ReceivableProtocol using read() or recv() as appropriate. Nr'cCsD||_|dur ||_n||_t|_d|_t|_t|_||_ dSNr) rFrr rp_offsetr_rbufr_trailer _zlib_bufsize)r,rFrrMr-r-r.r/s zPackStreamReader.__init__cs||}t|}j|7_tj}|dkr|}d}n t||dd}|}jttfddt|Dj || dj|d| |S)aHRead up to size bytes using the given callback. As a side effect, update the verifier's hash (excluding the last 20 bytes read). Args: read: The read callback to read from. size: The maximum number of bytes to read; the particular behavior is callback-specific. r'rcsg|]}jqSr-)r\popleft)rrrWr-r.rsz*PackStreamReader._read..N) r_rZr\maxrprrD bytearrayrextend)r,r[rrntnto_popto_addr-rWr._reads  zPackStreamReader._readcCs4|j}|}|dt|}||||SrY)r[tellseekr)r,bufrrr-r-r._buf_lens   zPackStreamReader._buf_lencCs|j|Sr))rZrjrWr-r-r.rnrzPackStreamReader.offsetcCsD|}||kr|j|S|j}t|_|||j||S)z)Read, blocking until size bytes are read.)rjr[r[rrfrF)r,rbuf_lenbuf_datar-r-r.r[s   zPackStreamReader.readcCs:|}|r|j|}||krt|_|S||j|S)z7Read up to size bytes, blocking until one byte is read.)rjr[r[rrfr)r,rrkrr-r-r.recvs zPackStreamReader.recvcCrr) _num_objectsrWr-r-r.rYrzPackStreamReader.__len__Fc cst|j\}|_t|jD]0}|j}t|j|j||jd\}}||_t}| || |j | d||_ |Vq| dkrJ|dt |j}||jkr`tt||jdS)aRead the objects in this pack file. Args: compute_crc32: If True, compute the CRC32 of the compressed data. If False, the returned CRC32 will be None. Returns: Iterator over UnpackedObjects with the following members set: offset obj_type_num obj_chunks (for non-delta types) delta_base (for delta types) decomp_chunks decomp_len crc32 (if compute_crc32 is True) Raises: ChecksumMismatch: if the checksum of the pack contents does not match the checksum in the pack trailer. zlib.error: if an error occurred during zlib decompression. IOError: if an error occurred writing to the output file. )rrLrMrr'N)rCr[rorrnrTrmr]rwriter[rhrjr`r\rpr!rr"r) r,rL pack_versionrrnrrripack_shar-r-r. read_objectss.      zPackStreamReader.read_objectsr1F)r2r3r4r5 _ZLIB_BUFSIZEr/rfrjr.rnr[rmrZrYr rIrsr-r-r-r.rXs     rXcs<eZdZdZd d fdd ZfddZd d dd ZZS) PackStreamCopierzClass to verify a pack stream as it is being read. The pack is read from a ReceivableProtocol using read() or recv() as appropriate and written out to the given file-like object. Nr'cs tj||d||_||_dS)aInitialize the copier. Args: read_all: Read function that blocks until the number of requested bytes are read. read_some: Read function that returns at least one byte, but may not return the number of bytes requested. outfile: File-like object to write output through. delta_iter: Optional DeltaChainIterator to record deltas as we read them. )rN)r r/outfile _delta_iter)r,rFrrw delta_iterr r-r.r/Cs  zPackStreamCopier.__init__cst||}|j||S)z:Read data from the read callback and write it to the file.)r rfrwrp)r,r[rrr r-r.rfSs zPackStreamCopier._readcCszd}t|D]!\}}|jr|j||dur)|d|dt|ddq|dur;|d|dddSdS) zVerify a pack stream and write it to the output file. See PackStreamReader.iterobjects for a list of exceptions this may throw. rNzcopying pack entries: / rzcopied z pack entries )rIrsrxrecordr_r)r,r=rrr-r-r.verifyYs  zPackStreamCopier.verifyr)r1)r2r3r4r5r/rfr}r/r-r-r r.rv<s  rvcCsRt}|t|t|t|tr|||S|D]}||q|S)z5Compute the SHA for a numeric type and object chunks.)r rr!rErwrDr!)typerDrpchunkr-r-r.rtis   rtcCst}|dt|}|dkr|||ks||kr(td|d|d||||}|||rJ|t||}|||t|8}|s5|S)aHash a portion of a file into a new SHA. Args: f: A file-like object to read from that supports seek(). start_ofs: The offset in the file to start reading at. end_ofs: The offset in the file to end reading at, relative to the end of the file. buffer_size: A buffer size for reading. Returns: A new SHA object updated with data read from the file. rz/Attempt to read beyond file length. start_ofs: z , end_ofs: z, file length: ) r rhrrgr7r[minrr_)r start_ofsend_ofsrrplengthtodorr-r-r.compute_file_shaus     rc@s>eZdZdZd7d8ddZeddZedd Zed9d d Z ed d Z d8ddZ ddZ ddZ ddZddZdefddZddZdddefdd Z d7d!eefd"d#Z  d7d$eed!eefd%d&Zd7d'd(Zd7d)d*Zd:d,d-Zd.d/Zd8d0d1Zddd2ededefd3d4Z d2ede!ee"ffd5d6Z#dS);PackDataaCThe data contained in a packfile. Pack files can be accessed both sequentially for exploding a pack, and directly with the help of an index to retrieve a specific object. The objects within are either complete or a delta against another. The header is variable length. If the MSB of each byte is set then it indicates that the subsequent byte is still part of the header. For the first byte the next MS bits are the type, which tells you the type of object, and whether it is a delta. The LS byte is the lowest bits of the size. For each subsequent byte the LS 7 bits are the next MS bits of the size, i.e. the last byte of the header contains the MS bits of the size. For the complete objects the data is stored as zlib deflated data. The size in the header is the uncompressed object size, so to uncompress you need to just keep feeding data to zlib until you get an object back, or it errors on bad data. This is done here by just giving the complete buffer from the start of the deflated object on. This is bad, but until I get mmap sorted out it will have to do. Currently there are no integrity checks done. Also no attempt is made to try and detect the delta case, or a request for an object at the wrong position. It will all just throw a zlib or KeyError. Nr'cCsd||_||_d|_|durt|jd|_n||_t|jj\}|_tt t t t ffdt d|_ dS)aaCreate a PackData object representing the pack in the given filename. The file must exist and stay readable until the object is disposed of. It must also stay the same size. It will be mapped whenever needed. Currently there is a restriction on the size of the pack as the python mmap implementation is flawed. r>Nri@) compute_size)rr _header_sizerrrCr[rorrZrBrrW _offset_cache)r,rrrrr-r-r.r/s  zPackData.__init__cCstj|jSr))rrbasenamerrWr-r-r.rrzPackData.filenamecCrr)r rWr-r-r.rr z PackData.pathcCs|t|||dS)N)rr)rC)rrrr-r-r. from_filerzPackData.from_filecCs ||dS)N)rr-)rrr-r-r. from_pathrzPackData.from_pathcCs|jdSr))rrrWr-r-r.rszPackData.closecC|Sr)r-rWr-r-r. __enter__rzPackData.__enter__cC |dSr)rr,exc_typeexc_valexc_tbr-r-r.__exit__rzPackData.__exit__cCst|tr ||kSdSr )rwrrrr-r-r.r~s zPackData.__eq__cCsV|jdur|jStj|j|_|j|jkr(|jd|jd|jd}t||jS)Nz is too small for a packfile (z < ))rrrgetsizerrr7)r,errmsgr-r-r. _get_sizes  zPackData._get_sizecCr)z+Returns the number of objects in this pack.rnrWr-r-r.rYr zPackData.__len__cCst|jddS)z[Calculate the checksum for this pack. Returns: 20-byte binary SHA1 digest r r)rrr!rWr-r-r.rr$zPackData.calculate_checksumFrGrHccsr|j|j|jdurdSt|jD]"}|j}t|jjd|d\}}||_|V|jt | t qdS)NF)rLrH) rrhrrorrgrTr[rnr_r)r,rHrrnrrr-r-r. iter_unpackeds    zPackData.iter_unpackedresolve_ext_refccsD|j}tj||d}t|D]\}}|dur||||VqdS)Yield entries summarizing the contents of this pack. Args: progress: Progress function, called with current and total object count. Returns: iterator of tuples with (sha, offset, crc32) rN)ro PackIndexer for_pack_datarI)r,r=rrBindexerrresultr-r-r.rs  zPackData.iterentriesr=cCst|j||dSzReturn entries in this pack, sorted by SHA. Args: progress: Progress function, called with current and total object count Returns: Iterator of tuples with (sha, offset, crc32) r=r)sortedr)r,r=rr-r-r.rs zPackData.sorted_entriescCL|j||d}t|d}t|||WdS1swYdS)zCreate a version 1 file for this data file. Args: filename: Index filename. progress: Progress report function Returns: Checksum of index file rwbN)rrwrite_pack_index_v1rr,rr=rrrr-r-r.create_index_v1#  $zPackData.create_index_v1cCr)zCreate a version 2 index file for this data file. Args: filename: Index filename. progress: Progress report function Returns: Checksum of index file rrN)rrwrite_pack_index_v2rrr-r-r.create_index_v21rzPackData.create_index_v2rcCs>|dkr |j|||dS|dkr|j|||dStd|)zCreate an index file for this data file. Args: filename: Index filename. progress: Progress report function Returns: Checksum of index file rrrzunknown index format )rrr)r,rr=rrr-r-r. create_index?szPackData.create_indexcCs|jdt|jdS)z1Return the expected checksum stored in this pack.r r')rrhrr[rWr-r-r.rRs zPackData.get_stored_checksumcCr)z#Check the consistency of this pack.Nrrr-r-r.rWrzPackData.checkrncCs8||jksJ|j|t|jj|d\}}||_|S)z5Given offset in the packfile return a UnpackedObject.rG)rrrhrTr[rn)r,rnrHrrr-r-r.get_unpacked_object_at^s  zPackData.get_unpacked_object_atcCs<z|j|WStyYnw|j|dd}|j|fS)zGiven an offset in to the packfile return the object that is there. Using the associated index the location of an object can be looked up, and then the packfile can be asked directly for that object using this function. FrG)rrrrorx)r,rnrr-r-r. get_object_aths  zPackData.get_object_atNNr1r))NrN)$r2r3r4r5r/r.rrrrrrrrr~rrZrYrrErrResolveExtRefFnr ProgressFnrrrrrrrIrrBrrr-r-r-r.rsX              rTc@seZdZdZdZdZddd*ddZed+defd d Z eddd d d de e de fddZ deddfddZdeddfddZddZd*ddZddZdedefddZded ed!ee defd"d#Zded ed!ee fd$d%Zdeefd&d'Zd(d)ZdS),DeltaChainIteratoraAbstract iterator over pack data based on delta chains. Each object in the pack is guaranteed to be inflated exactly once, regardless of how many objects reference it as a delta base. As a result, memory usage is proportional to the length of the longest delta chain. Subclasses can override _result to define the result type of the iterator. By default, results are UnpackedObjects with the following members set: * offset * obj_type_num * obj_chunks * pack_type_num * delta_base (for delta types) * comp_chunks (if _include_comp is True) * decomp_chunks * decomp_len * crc32 (if _compute_crc32 is True) FNrr'cCs0||_||_tt|_tt|_g|_g|_dSr))r_resolve_ext_refrr _pending_ofs _pending_ref _full_ofs _ext_refs)r,file_objrr-r-r.r/s    zDeltaChainIterator.__init__ pack_datacCs6|d|d}|||jddD]}||q|S)NrFrG) set_pack_datarr|)rrrwalkerrr-r-r.rs   z DeltaChainIterator.for_pack_datarNrrPackr+rNc Cs*|d|d}||jt}|D]#}t|tsJz|j|}Wn ty.|s,Yqw||qt} |r| }|j |} | | | |d} | j t kr\| j| j} n(| j tkrttt| jtsnJ|j| j} Wdn1swY| dur| | vr|| |s:|S)Nr)rrrTrwrDrrrrpoprr|rorJrnrkrKrr) rrr+rNrrrrpoffdonerbase_ofsr-r-r.for_pack_subsets>          z"DeltaChainIterator.for_pack_subsetrcCst|j}|j}|tkr||j}|j||dS|tkr0t|jts%J|j |j|dS|j ||fdSr)) rornrJrkrrarKrwrDrr)r,rrOrn base_offsetr-r-r.r|s zDeltaChainIterator.recordcCs |j|_dSr))r)r,rr-r-r.rrz DeltaChainIterator.set_pack_dataccsL|jD]\}}|||dEdHq|EdH|jr$Jt|jdSr))r _follow_chain_walk_ref_chainsrrepr)r,rnrOr-r-r._walk_all_chainss z#DeltaChainIterator._walk_all_chainscCs|jr tdd|jDdS)NcSsg|]}t|qSr-)r"rr-r-r.rz9DeltaChainIterator._ensure_no_pending..)rr&rWr-r-r._ensure_no_pendingsz%DeltaChainIterator._ensure_no_pendingc cs|js |dSt|jD]8\}}||jvrqz ||\}}Wn ty-Yqw|j||j||D] }| |||EdHqs  z3DeltaChainIterator._follow_chain..) rrrr rrnrrpra)r,rnrmrr unblockedr-rr.r s   z DeltaChainIterator._follow_chaincCs|Sr))rrWr-r-r.rVzDeltaChainIterator.__iter__cCrr))rrWr-r-r.ext_refs!rzDeltaChainIterator.ext_refsr1r))r2r3r4r5rrr/rrrr rDrErrIr|rrrrrrrZrrrr rVrr-r-r-r.rzsF %    rc@eZdZdZddZdS)UnpackedObjectIteratorz1Delta chain iterator that yield unpacked objects.cCs|Sr)r-rr-r-r.r(rzUnpackedObjectIterator._resultNr2r3r4r5rr-r-r-r.r% rc@seZdZdZdZddZdS)rz/Delta chain iterator that yields index entries.TcCs||j|jfSr))rprnr\rr-r-r.r1zPackIndexer._resultN)r2r3r4r5rrr-r-r-r.r,s rc@r) PackInflaterz1Delta chain iterator that yields ShaFile objects.cCs|Sr))rvrr-r-r.r8rzPackInflater._resultNrr-r-r-r.r5rrc@sHeZdZdZdddZdddZdd eddfd d Zd d ZddZ dS) SHA1ReaderAWrapper for file-like object that remembers the SHA1 of its data.r'NcCs||_td|_dSN)rr r,rr-r-r.r/?szSHA1Reader.__init__cCs|j|}|j||Sr))rr[r r)r,rVrr-r-r.r[Cs  zSHA1Reader.readF allow_emptycCsF|jd}||jkr|rt|dkr!t|jt|dSdS)Nr's(0000000000000000000000000000000000000000)rr[r r!r"rr)r,rrr-r-r. check_shaHs  zSHA1Reader.check_shacC |jSr))rrrWr-r-r.rQr0zSHA1Reader.closecCrr)rrgrWr-r-r.rgTr0zSHA1Reader.tellr1r)rt) r2r3r4r5r/r[rErrrgr-r-r-r.r<s   rc@sDeZdZdZdddZdddZdd Zd d Zd d ZddZ dS) SHA1Writerrr'NcCs||_d|_td|_dS)Nrr)rrr rr-r-r.r/[szSHA1Writer.__init__cCs.|j||j||jt|7_dSr))r rrrprr_rr-r-r.rp`s  zSHA1Writer.writecCs<|j}t|dks J|j||jt|7_|Sr<)r r!r_rrprrr-r-r. write_shaes  zSHA1Writer.write_shacCs|}|j|Sr))rrrrr-r-r.rls zSHA1Writer.closecCrr))rrWr-r-r.rnqrzSHA1Writer.offsetcCrr)rrWr-r-r.rgtr0zSHA1Writer.tellr1) r2r3r4r5r/rprrrnrgr-r-r-r.rXs   rcCsg}|d>|d@B}|dL}|r!||dB|d@}|dL}|s|||tkrR|d@g}|dL}|rI|d8}|dd|d@B|dL}|s5||t|S|tkrbt|dks^J||7}t|S) a Create a pack object header for the given object info. Args: type_num: Numeric type of the object. delta_base: Delta base offset or ref, or None for whole objects. size: Uncompressed object size. Returns: A header for a packed object. rrGr^rHr$rrr')rarJinsertrarKr_r`)rOrkrrAcrcr-r-r.pack_object_headerxs.    rr]ccst|tvr |\}}nd}t|tr|g}tt||ttt|Vtj|d}|D]}| |Vq*| VdS)zGenerate chunks for a pack object. Args: type: Numeric type of the object object: Object to write compression_level: the zlib compression level Returns: Chunks N)level) rsrwrDrrqrrr_r compressobjcompressflush)r~objectcompression_levelrk compressorrr-r-r.pack_object_chunkss   rcCsFd}t|||dD]}|||dur||t||}q |d@S)aWrite pack object to a file. Args: write: Write function to use type: Numeric type of the object object: Object to write compression_level: the zlib compression level Returns: Tuple with offset at which the object was written, and crc32 rrNr)rrr`r\)rpr~rrprr\rr-r-r.write_pack_objects  r)deltifydelta_window_sizerr<rrrcCst|dd}t|j||||d\}}Wdn1swYtdd|D}t|dd}|t|||fWdS1sHwYdS)aWrite a new pack data file. Args: filename: Path to the new pack file (without .pack extension) container: PackedObjectContainer entries: Sequence of (object_id, path) tuples to write delta_window_size: Delta window size deltify: Whether to deltify pack objects compression_level: the zlib compression level Returns: Tuple with checksum of pack file and index file .packrrrrNcSs"g|] \}}||d|dfqS)rrr-)rkvr-r-r.rs"zwrite_pack...idx)rwrite_pack_objectsrprrr)rr<rrrrrdata_sumr-r-r. write_packs $rccs(dVtddVtd|VdS)zYield chunks for a pack header.r?rrN)rr)rBr-r-r.pack_header_chunkssrcCs:t|dr|j}tjdtddt|D]}||qdS)z4Write a pack header for the given number of objects.rpz?write_pack_header() now takes a write rather than file argumentrrN)hasattrrprrrr)rprBrr-r-r.write_pack_headers   r other_havesr= container object_idsr ccs|durt}d}t|j|dddD]6\}}|dur1|ddkr1|d|dt|d|jtkrJt|j}||vsC||vrJ|V|d7}q|dur[|d |d dSdS) NrT)rNrQzchecking for reusable deltas: rzr{rzfound z deltas to reuse ) rTrIrRr_rrorKr"rk)r r r r=reusedrrhexshar-r-r.find_reusable_deltass(   r window_sizer=rc#s,fdd}tt|||dEdHdS)aGenerate deltas for pack objects. Args: objects: An iterable of (object, path) tuples to deltify. window_size: Window size; None for default Returns: Iterator over type_num, object id, delta_base, content delta_base is None for full text entries c3sHD]}t|tr||jdffVq|d|dj|dffVqdS)Nrr)rwrrO)er<r-r.objects_with_hintss  z0deltify_pack_objects..objects_with_hintsrN)deltas_from_sorted_objectssort_objects_for_delta)r<rr=rr-rr.deltify_pack_objectss rcCsng}|D]'}t|tr|\}}|durd}d}n|\}}n|}|||| |fq|dd|DS)Ncss|]}|dVqdS)r@Nr-)rxr-r-r.r=sz)sort_objects_for_delta..)rwrBra raw_lengthsort)r<magicentryr7hintrOrr-r-r.r+s  rccs4|durt}t}t|D]\}}|dur&|ddkr&|d|d|}|}ttt|}d} |D]3\} } } | |jkrBq7d} g}t | |D]}| t|7} | |krYn| |qK| } |}ttt|}q7t |j| | ||dV|| |j|ft||kr|t||ksqdS)Nr rzgenerating deltas: r{)rprkrjri)DEFAULT_PACK_DELTA_WINDOW_SIZErrIr as_raw_chunksrqrrr_rO create_deltararIrpr! appendleftr)r<rr=possible_basesrorNwinner winner_len winner_basebase_id base_type_numbase delta_lendeltarr-r-r.r@sL      r)rr ofs_deltar=r-csDt}|dur d}|r|tt||dfSfdd}||fS)Create pack data from objects. Args: objects: Pack objects Returns: Tuples with (type_num, hexdigest, delta base, object chunks) NFrc3s4D]}t|trt|dVqt|VqdSrY)rwrBfull_unpacked_objectr$rr-r.iter_without_paths   z/pack_objects_to_data..iter_without_path)r_rr)r<rrr-r=countr1r-rr.pack_objects_to_datais r3 reuse_deltasc #st||rt|t||dD] }t|=|Vq|dur$d}|rD|jdd} ttfdd| D||dEdHdSD] } t || VqFdS)r.r NFrMc3s|] }||jfVqdSr)idrr$rr-r.rsz,generate_unpacked_objects..r) dictrrTr"rprOkeysrrr/) r r rrr4r-r r=robjects_to_deltaoidr-r8r.generate_unpacked_objectss,  r=r$cCs t|jdd||dS)N)rkr\rirp)rIrOr rpr!r0r-r-r.r/s r/c Cs,t|}t||||||d} t|| ||dS)aWrite a new pack data file. Args: write: write function to use container: PackedObjectContainer entries: Sequence of (object_id, path) tuples to write delta_window_size: Sliding window size for searching for deltas; Set to None for default window size. deltify: Whether to deltify objects compression_level: the zlib compression level to use Returns: Dict mapping id -> (offset, crc32 checksum), pack checksum )rrr4r  num_recordsr)r_r=write_pack_data) rpr r rrr4rr pack_contents_count pack_contentsr-r-r.write_pack_from_containers rCrcCs t||d\}}t||||dS)aWrite a new pack data file. Args: write: write function to use objects: Sequence of (object, path) tuples to write delta_window_size: Sliding window size for searching for deltas; Set to None for default window size. deltify: Whether to deltify objects compression_level: the zlib compression level to use Returns: Dict mapping id -> (offset, crc32 checksum), pack checksum )rr>)r3r@)rpr<rrrrArBr-r-r.rsrc @sbeZdZ     d dddZddZd d Zddddd d eed ede dee fddZ dS)PackChunkGeneratorNr]Tr'cCs*td|_i|_|j|||||d|_dS)Nr)r?recordsr=rreuse_compressed)r csr_pack_data_chunks_it)r,r?rEr=rrFr-r-r.r/ s  zPackChunkGenerator.__init__cCrr))rGr!rWr-r-r. sha1digestr0zPackChunkGenerator.sha1digestcCrr))rIrWr-r-r.rV rzPackChunkGenerator.__iter__)r?r=rrFrErrFc cs|dur t|}d}t|D]}|V|j||t|7}qd}t|D]\} } | j} |durF| ddkrF|d| d|dd| jduryz |j| j\} } Wnt ynt } t | jt sfJ| j| j f}Ynwt} || | j f}n| j }| jdur|r| j}nt| ||d}d}d}|D]}|Vt||}|j||t|7}q|d 7}||f|j| <||7}q'||krtd |d ||jVdS) arIterate pack data file chunks. Args: records: Iterator over UnpackedObject num_records: Number of records (defaults to len(records) if not specified) progress: Function to report progress to compression_level: the zlib compression level Returns: Dict mapping id -> (offset, crc32 checksum), pack checksum Nrr zwriting pack data: rzr{rrrz actual records written differs: z != )r_rrGrrIrorrkrrrKrwrDrirJrhrr`r\rpr7r!)r,rEr?r=rrFrnractual_num_recordsrrrOr base_crc32rNrDr\ object_sizer-r-r.rH#sX       z$PackChunkGenerator._pack_data_chunks)NNNr]Tr1) r2r3r4r/rJrVr rIrZrErDrHr-r-r-r.rD s0 rD)r?r=rrEcCs0t||||d}|D]}||q |j|fS)aWrite a new pack data file. Args: write: Write function to use num_records: Number of records (defaults to len(records) if None) records: Iterator over type_num, object_id, delta_base, raw progress: Function to report progress to compression_level: the zlib compression level Returns: Dict mapping id -> (offset, crc32 checksum), pack checksum )r?rEr=r)rDrrJ)rprEr?r=rchunk_generatorrr-r-r.r@gs r@cCst|}tdd}|D]\}}}|t|ddd7<q tdD]}|td||||d||7<q$|D]\}}}|dksMtd|td ||q@t|d ks`J||| S) aWrite a new pack index file. Args: f: A file-like object to write to entries: List of tuples with object name (sha), offset_in_pack, and crc32_checksum. pack_checksum: Checksum of the pack file. Returns: The SHA of the written index file cSr8rYr-r-r-r-r.r;z%write_pack_index_v1..Nrrrrz)pack format 1 only supports offsets < 2Gbr0r') rrrbrrprr TypeErrorr_r)rrr fan_out_tablerrnentry_checksumrr-r-r.rs   rcCsNt}|d@}|dL}|r||dB|d@}|dL}|s ||t|S)NrHr$r^)r`rarD)rrcrr-r-r._delta_encode_sizes rSicCstdg}tdD]}|d|d>@r(|||d?d@|dd|>O<q tdD]!}|d|d>@rN|||d?d@|ddd|>O<q-t|S)Nr^rrrrr)r`rrarD)rrscratchrr-r-r._encode_copy_operations   rVc csVt|tr d|}t|trd|}t|tsJt|ts#Jtt|Vtt|Vtd||d}|D]l\}}}}}|dkrh|}||} | dkrht| t } t || V|| 7}| | 8} | dksQ|dksp|dkr||} |} | dkrtdgVt || | dV| d8} | d7} | dkszt| gVt || | | Vq@ryt|||d} |d7}| | | d >O} q[d} td D] } |dd| >@rt|||d} |d7}| | | d >O} q| dkrd } | | | ks| | |ks| |krn(||| | | n|dkr|||||||7}ntd ||ksC||krtd ||d|t|krtd|S)zBased on the similar function in git's patch-delta.c. Args: src_buf: Source buffer delta: Delta instructions rrcSs^d}d}|r+t|||d}|d7}||d@|>O}|d7}|d@s) ||fS|s||fS)Nrrir$r^)rb)r,rrrcmdr-r-r.get_delta_header_sizesz*apply_delta..get_delta_header_sizez vs rr^rrr@rzInvalid opcode 0zdelta not empty: Nzdest size incorrect) rwrDrr_rbrrarrE) src_bufr,outr delta_lengthrjsrc_size dest_sizericp_offrrcp_sizer-r-r.rs\     "        rrrc Csft|}|d|tddtdd}|D]\}}}|t|ddd7<qg}tdD]}|td ||||d||7<q4|D] \}}}||qP|D]\}}}|td |q]|D]%\}}}|d kr|td |qn|td d t|||qn|D] }|td |qt|d ksJ||| S) aWrite a new pack index file. Args: f: File-like object to write to entries: List of tuples with object name (sha), offset_in_pack, and crc32_checksum. pack_checksum: Checksum of the pack file. Returns: The SHA of the index file written rrrcSr8rYr-r-r-r-r.rOB r;z%write_pack_index_v2..Nrrrr=s>Qr') rrprrrrbrr_rar) rrrrQrrnrR largetablerr-r-r.r3 s0      rc @seZdZUdZeegefed<eegefed<eeed<eeed< dTdee ddfd d Z e d d Z e d dZ ddZedefddZedefddZdUddZddZddZddZdefddZdefdd Zd!d"ZdUd#d$ZdUd%d&Zdefd'd(Zd)d*Zd+ede fd,d-Z!d+ede"eeffd.d/Z#d+ede$fd0d1Z%de&e$fd2d3Z'd4d5d6e(e)d7e de&e$fd8d9Z*d4d4d4d:d6e(e)d;e d7e dZ,dVd?d@Z-dTdAeedefdBdCZ.dDede"eeee/ffdEdFZ0 dTdGedHede"ee(effdIdJZ1 dTdKee2de&e3fdLdMZ4 dTdKee2de&e3fdNdOZ5d4dPdQdDed;e d.crwr))r _idx_pathr-rWr-r.rOp ry) _basenamerurvrzrxrsrtr)r,rrr-rWr.r/g s   z Pack.__init__cCs|d}||_||_|S)z]Create a new pack object from callables to load pack data and index objects. )rsrt)rdata_fnidx_fnrcr-r-r.from_lazy_objectss szPack.from_lazy_objectscCs,|d}||_d|_||_d|_||S)z:Create a new pack object from pack data and index objects.r|N)rursrvrtcheck_length_and_checksum)rrr(rcr-r-r. from_objects} szPack.from_objectscCr)z2The SHA over the SHAs of the objects in this pack.)rrrWr-r-r.r rz Pack.namecCs,|jdur|js J||_||jS)z The pack data object being used.N)rursrrWr-r-r.r s   z Pack.datacCs$|jdur|js J||_|jS)zLThe index being used. Note: This may be an in-memory index N)rvrtrWr-r-r.r s   z Pack.indexcCs0|jdur |j|jdur|jdSdSr))rurrvrWr-r-r.r s   z Pack.closecCrr)r-rWr-r-r.r rzPack.__enter__cCrr)rrr-r-r.r rz Pack.__exit__cCst|t|o |j|jkSr))rwr~rrr-r-r.r~ sz Pack.__eq__cCr)zNumber of entries in this pack.)r_rrWr-r-r.rY rz Pack.__len__cCs|jjd|jdS)N(r)rr2r{rWr-r-r.r sz Pack.__repr__cCr)z7Iterate over all the sha1s of the objects in this pack.)rrrWr-r-r.rV rz Pack.__iter__cCsht|jt|jksJdt|jdt|jd|j}|j}||kr2tt|t|dS)z@Sanity check the length and checksum of the pack index and data.zLength mismatch: z (index) != z (data)N)r_rrrrrr")r,idx_stored_checksumdata_stored_checksumr-r-r.r s  zPack.check_length_and_checksumcCs.|j|j|D]}|qdS)zCheck the integrity of this pack. Raises: ChecksumMismatch: if a checksum for the index or data is wrong N)rrr iterobjectsr9r-r-r.r s    z Pack.checkcCrr))rrrWr-r-r.r r0zPack.get_stored_checksumcCsdd|DS)NcSsg|]}|dfqSr)r-r7r-r-r.r rz$Pack.pack_tuples..)rrWr-r-r. pack_tuples rzPack.pack_tuplesr cCs(z |j|WdStyYdSw)z3Check whether this pack contains a particular SHA1.TF)rrrr?r-r-r.r@ s   zPack.__contains__cCs<|j|}|j|\}}||||\}}|d|fSr)rrrrresolve_objectr)r,r rnobj_typer7rOrDr-r-r.get_raw s z Pack.get_rawcCs||\}}tj|||dS)zRetrieve the specified SHA1.)rp)rrfrom_raw_string)r,r r~uncompr-r-r.rA szPack.__getitem__cCsttj|j|jdS)z&Iterate over the objects in this pack.r)rrrrrrWr-r-r.r szPack.iterobjectsFrMr+rNcs"fddtj|||jdDS)Nc3s|] }|jvr|VqdSr)r5)ruor*r-r.r s z*Pack.iterobjects_subset..r)rrrrPr-r*r.rO s zPack.iterobjects_subset)rHrNrQrHrQc cstt}i}t|}|j|dD]\}|} | ||j<t| } | |vrZ|jtkrRt |j t s1J|j|j } z|| |_ Wnt yN||  |Yqwt|_|V|| ||jgD] } t| _| | _ | Vqaq|rrJ|sz|r|t|dSdS)NrG)rrrTrrprnr"rorJrwrkrZrrarKremoverr&) r,r+rHrNrQ ofs_pendingofsrrrprrchildr-r-r.rR s<     zPack.iter_unpacked_subsetccsNdd|jD}|jj|dD]}||j\}}||_||_|VqdS)NcSsi|] \}}}|||fqSr-r-)rrprr\r-r-r. % sz&Pack.iter_unpacked..rG)rrrrrnrgr\)r,rHofs_to_entriesrrpr\r-r-r.r$ szPack.iter_unpackedmsgcCsd|jd}t|d}|r |||dWd|SWd|S1s+wY|S)a'Add a .keep file for the pack, preventing git from garbage collecting it. Args: msg: A message written inside the .keep file; can be used later to determine whether or not a .keep file is obsolete. Returns: The path of the .keep file, as a string. z.keepr N)r{rrp)r,r keepfile_namekeepfiler-r-r.keep. s      z Pack.keeprpcCsfz|j|}Wn tyd}Ynw|r|j|\}}n|jr*||\}}nt||||fS)z8Get the object for a ref SHA, only looking in this pack.N)rrrrrr)r,rprnr~r7r-r-r.get_ref= s  z Pack.get_refrnr~cCs|}|}|}g}|tvrj|} |dur|j}|tkr1|\} } || }|j|\}}t|ts0Jn-|tkr^|\} } t| trDt | dksFJ|| \}}}t|tsTJ|| kr^t t | | | || f|tvs |} t |D]\} }} t| | } | dur|| f|jj| <qp|| fS)z|Resolve an object, possibly resolving deltas when necessary. Returns: Tuple with object type and contents. Nr')rsrrJrrrwrZrKrDr_r&r"rareversedrr)r,rnr~r7rr base_typebase_obj delta_stack prev_offset delta_offsetr,rrD delta_typer-r-r.rL s:   zPack.resolve_objectr=cC|jj||jdS)rr)rrrr,r=r-r-r.rz  z Pack.entriescCrr)rrrrr-r-r.r rzPack.sorted_entriesT)rHrQcCsX|j|}|jj||d}|jtkr*|r*t|jtsJ|j ||j|_t |_|S)zGet the unpacked object for a sha. Args: sha: SHA of object to fetch include_comp: Whether to include compression data in UnpackedObject rG) rrrrrorJrwrkrZrrK)r,rprHrQrnrr-r-r.rL s zPack.get_unpacked_objectr)r1rt)7r2r3r4r5rrrrrrr/rrrrr.rrrrrr~rZrYrCrrVrrrDrrrEr@rBrrrAr rr rrOrIrRrrrrrrrrrrLr-r-r-r.r^ s             $  /  r)rr=rcCs|dt|j\}}|r!|dt|j|t||t|dd}|dtj g}t |D]<\} } |durL|d| dt|d dt| d ksTJ|| \} } | } t |j| | ||d }|| | |fq4|}||||fS) zExtend a pack file with more objects. The caller should make sure that object_ids does not contain any objects that are already in the pack rr rNzwriting extra base objects: rzr{rr')rpr)rhrCr[rrpr_rrrrrIrrgrrar!)rr rrr=_versionrBnew_sha extra_entriesr object_idrOrrnr\rrr-r-r. extend_pack s8     r)rrr))rrr)r])Nr]r1r)NNTTNN)NNTr]N)r5r` collectionsrr contextlibriorrcdifflibrrModuleNotFoundErrordifflibrrsysrrcollections.abcr r r hashlibr itertoolsr rrrtypingrrrrrrrr ImportErrorrplatformerrorsrrrr lru_cacherr<rrr r!r"rJrKrsrPACK_SPOOL_FILE_MAX_SIZErBrDrZrrrrCrPackHint Exceptionr&r6rFrUrerfrIrurErrrrrrrrrrrrrCrErTrWrXrvrtrrrrrrrrrrrrrrrrTrrrrr3r=r/rCrrDr@rrSr\rVr!rrwrite_pack_indexrr dulwich._packr-r-r-r.s      $     $   d  D ]/8)    M - h,          , ,  )  , a  .G (Q  6