o h2@sddlZddlmZmZmZmZddlmZmZm Z m Z ddl Z gdZ e dddZ Gdd d ee ZGd d d eeZGd d d eeZGdddeeZGdddeeZGdddeeeZdS)N)IterableIteratorSequenceSized)GenericOptionalTypeVarUnion) BatchSampler RandomSamplerSamplerSequentialSamplerSubsetRandomSamplerWeightedRandomSampler_T_coT) covariantc@s:eZdZdZd deeddfddZdeefddZ dS) r a;Base class for all Samplers. Every Sampler subclass has to provide an :meth:`__iter__` method, providing a way to iterate over indices or lists of indices (batches) of dataset elements, and may provide a :meth:`__len__` method that returns the length of the returned iterators. Args: data_source (Dataset): This argument is not used and will be removed in 2.2.0. You may still have custom implementation that utilizes it. Example: >>> # xdoctest: +SKIP >>> class AccedingSequenceLengthSampler(Sampler[int]): >>> def __init__(self, data: List[str]) -> None: >>> self.data = data >>> >>> def __len__(self) -> int: >>> return len(self.data) >>> >>> def __iter__(self) -> Iterator[int]: >>> sizes = torch.tensor([len(x) for x in self.data]) >>> yield from torch.argsort(sizes).tolist() >>> >>> class AccedingSequenceLengthBatchSampler(Sampler[List[int]]): >>> def __init__(self, data: List[str], batch_size: int) -> None: >>> self.data = data >>> self.batch_size = batch_size >>> >>> def __len__(self) -> int: >>> return (len(self.data) + self.batch_size - 1) // self.batch_size >>> >>> def __iter__(self) -> Iterator[List[int]]: >>> sizes = torch.tensor([len(x) for x in self.data]) >>> for batch in torch.chunk(torch.argsort(sizes), len(self)): >>> yield batch.tolist() .. note:: The :meth:`__len__` method isn't strictly required by :class:`~torch.utils.data.DataLoader`, but is expected in any calculation involving the length of a :class:`~torch.utils.data.DataLoader`. N data_sourcereturncCs"|durddl}|ddSdS)Nrzz`data_source` argument is not used and will be removed in 2.2.0.You may still have custom implementation that utilizes it.)warningswarn)selfrrrl/var/www/html/construction_image-detection-poc/venv/lib/python3.10/site-packages/torch/utils/data/sampler.py__init__@s zSampler.__init__cCstN)NotImplementedErrorrrrr__iter__IszSampler.__iter__r) __name__ __module__ __qualname____doc__rrrrrrrrrrr s) r c@sLeZdZUdZeed<deddfddZdeefddZ defd d Z dS) r z~Samples elements sequentially, always in the same order. Args: data_source (Dataset): dataset to sample from rrNcCs ||_dSr)r)rrrrrrq zSequentialSampler.__init__cCsttt|jSr)iterrangelenrrrrrrtszSequentialSampler.__iter__cC t|jSr)r%rrrrr__len__wr"zSequentialSampler.__len__) rrr r!r__annotations__rrintrr'rrrrr hs r c @szeZdZUdZeed<eed<   ddededeeddfdd Z e defd d Z de efd d Z defddZdS)r aSamples elements randomly. If without replacement, then sample from a shuffled dataset. If with replacement, then user can specify :attr:`num_samples` to draw. Args: data_source (Dataset): dataset to sample from replacement (bool): samples are drawn on-demand with replacement if ``True``, default=``False`` num_samples (int): number of samples to draw, default=`len(dataset)`. generator (Generator): Generator used in sampling. r replacementFN num_samplesrcCs^||_||_||_||_t|jtstd|jt|jtr%|jdkr-t d|jdS)N;replacement should be a boolean value, but got replacement=rDnum_samples should be a positive integer value, but got num_samples=) rr* _num_samples generator isinstancebool TypeErrorr+r) ValueError)rrr*r+r/rrrrs   zRandomSampler.__init__cCs|jdur t|jS|jSr)r.r%rrrrrr+s  zRandomSampler.num_samplesccst|j}|jdur#ttjdtjd}t }| |n|j}|j rWt |j dD]}tj|dtj|dEdHq0tj||j dftj|dEdHdSt |j |D]}tj||dEdHq^tj||dd|j |EdHdS)Nrdtype )r6)highsizer5r/r/)r%rr/r)torchemptyint64random_item Generator manual_seedr*r$r+randinttolistrandperm)rnseedr/_rrrrs6        zRandomSampler.__iter__cC|jSrr+rrrrr'zRandomSampler.__len__)FNN)rrr r!rr(r1rr)rpropertyr+rrr'rrrrr {s(   r c@sVeZdZUdZeeed<d deeddfddZdeefddZ defd d Z dS) rzSamples elements randomly from a given list of indices, without replacement. Args: indices (sequence): a sequence of indices generator (Generator): Generator used in sampling. indicesNrcCs||_||_dSr)rKr/)rrKr/rrrrs zSubsetRandomSampler.__init__ccs.tjt|j|jdD]}|j|Vq dSNr9)r:rCr%rKr/)rirrrrszSubsetRandomSampler.__iter__cCr&r)r%rKrrrrr'r"zSubsetRandomSampler.__len__r) rrr r!rr)r(rrrr'rrrrrs  rc @speZdZUdZejed<eed<eed<  dde e dededdfdd Z de efd d Z defd d ZdS)raNSamples elements from ``[0,..,len(weights)-1]`` with given probabilities (weights). Args: weights (sequence) : a sequence of weights, not necessary summing up to one num_samples (int): number of samples to draw replacement (bool): if ``True``, samples are drawn with replacement. If not, they are drawn without replacement, which means that when a sample index is drawn for a row, it cannot be drawn again for that row. generator (Generator): Generator used in sampling. Example: >>> # xdoctest: +IGNORE_WANT("non-deterministic") >>> list(WeightedRandomSampler([0.1, 0.9, 0.4, 0.7, 3.0, 0.6], 5, replacement=True)) [4, 4, 1, 4, 5] >>> list(WeightedRandomSampler([0.9, 0.4, 0.05, 0.2, 0.3, 0.1], 5, replacement=False)) [0, 1, 4, 3, 2] weightsr+r*TNrcCst|trt|ts|dkrtd|t|ts!td|tj|tjd}t|jdkr:tdt |j||_ ||_ ||_ ||_ dS)Nrr-r,r4z=weights should be a 1d sequence but given weights have shape )r0r)r1r3r: as_tensordoubler%shapetuplerNr+r*r/)rrNr+r*r/weights_tensorrrrrs.  zWeightedRandomSampler.__init__ccs2tj|j|j|j|jd}t|EdHdSrL)r: multinomialrNr+r*r/r#rB)r rand_tensorrrrrs zWeightedRandomSampler.__iter__cCrGrrHrrrrr'rIzWeightedRandomSampler.__len__)TN)rrr r!r:Tensorr(r)r1rfloatrrrr'rrrrrs$    rc@s^eZdZdZdeeeeefdededdfddZ de e efd d Z defd d Z dS) r aiWraps another sampler to yield a mini-batch of indices. Args: sampler (Sampler or Iterable): Base sampler. Can be any iterable object batch_size (int): Size of mini-batch. drop_last (bool): If ``True``, the sampler will drop the last batch if its size would be less than ``batch_size`` Example: >>> list(BatchSampler(SequentialSampler(range(10)), batch_size=3, drop_last=False)) [[0, 1, 2], [3, 4, 5], [6, 7, 8], [9]] >>> list(BatchSampler(SequentialSampler(range(10)), batch_size=3, drop_last=True)) [[0, 1, 2], [3, 4, 5], [6, 7, 8]] sampler batch_size drop_lastrNcCsXt|trt|ts|dkrtd|t|ts!td|||_||_||_dS)NrzBbatch_size should be a positive integer value, but got batch_size=z7drop_last should be a boolean value, but got drop_last=)r0r)r1r3rYrZr[)rrYrZr[rrrr-s   zBatchSampler.__init__ccstt|j}|jr|g|j}t|D]}g|VqdSgt||j}|r8|Vgt||j}|s(dSdSr)r#rYr[rZzip itertoolsislice)r sampler_iterargsbatch_droplastbatchrrrrFs     zBatchSampler.__iter__cCs0|jr t|j|jSt|j|jd|jS)NrO)r[r%rYrZrrrrr'TszBatchSampler.__len__)rrr r!r r r)rr1rrlistrr'rrrrr s r )r]collections.abcrrrrtypingrrrr r:__all__rr r)r r rrrcr rrrrs RKA