o oÇ hÙ-ã@s,dZddlZGdd„dƒZGdd„dƒZdS)aN Helper classes for working with low precision floating point types that align with the opencompute (OCP) microscaling (MX) specification. * MXFP4Tensor: 4-bit E2M1 floating point data * MXScaleTensor: 8-bit E8M0 floating point data Reference: https://www.opencompute.org/documents/ocp-microscaling-formats-mx-v1-0-spec-final-pdf éNc@s>eZdZddd„Zdd„Zdd„Zdd „Zd d „Zd d „ZdS)Ú MXFP4TensorNcCój||_|durt|tjƒsJdƒ‚|j|_| |¡|_dS|dur1t|tƒr+||_dS|f|_dStdƒ‚)at Tensor class for working with four bit E2M1 floating point data as defined by the opencompute microscaling specification. Parameters: - data: A torch tensor of float32 numbers to convert to fp4e2m1 microscaling format. - size: The size of the tensor to create. - device: The device on which to create the tensor. Nú%Parameter data must be a torch tensorú.Either parameter data or size must be provided© ÚdeviceÚ isinstanceÚtorchÚTensorÚ _from_floatÚdataÚtupleÚsizeÚ ValueError©Úselfr rr©rúe/var/www/html/construction_image-detection-poc/venv/lib/python3.10/site-packages/triton/tools/mxfp.pyÚ__init__s  zMXFP4Tensor.__init__cCsptjdd|jtj|jd}tjdd|jtj|jd}tjdd|jtj|jd}|d>|d>B|B tj¡|_|S)Nré©rÚdtyperééé)r ÚrandintrÚuint8rÚtyper )rÚSÚEÚMrrrÚrandom#s zMXFP4Tensor.randomcCs |tjks Jdƒ‚|j}|d?d@ |¡}|d?d@ |¡}|d@ |¡}t |¡}|dk|dk@}|}| ¡rs||} ||} ||} t d| ¡} t | dk| | d¡} t | dk| dd| d¡}| t d| ¡|}|||<|||dk@d9<| tj¡S) zŠ Convert fp4e2m1 data to float32. Returns: - A torch tensor of type dtype representing the fp4e2m1 data. zCCurrently only float32 is supported for fp4e2m1 to float conversionrrréÿÿÿÿçà?çð?r)r Úfloat32r rÚ zeros_likeÚanyÚpowÚwhere)rrr rrr ÚvalueÚis_zeroÚ non_zero_maskÚS_nzÚE_nzÚM_nzÚsignÚexponentÚmantissaÚvalue_nzrrrÚto+s&   zMXFP4Tensor.tocCsTt |¡ tj¡}t |¡}|dk}t |¡t |¡B}tjgd¢tj|jd}tjddgtj|jd}g}g} g} |D]M} | dkrcd} |D]} | d}|d| }|  |¡|   | ¡|   | ¡qFq<|   ¡d} |D]} d| d}|d| }|  |¡|   | ¡|   | ¡qkq| d>B| B tj¡S)a5 Convert float32 numbers to mxf4 e2m1 format. * No encodings are reserved for Inf or NaN in mxf4. * Conversion from float supports roundTiesToEven rounding mode. * If a value exceeds the mxf4 representable range after rounding, clamps to the maximum mxf4 magnitude, preserving the sign. * If a value has magnitude less than the minimum subnormal magnitude in mxf4 after rounding, converts to zero. Parameters: - values: A torch tensor of float32 numbers to convert to fp4 format. r)rrrr©rrrr#rr$r"T)ÚdimÚkeepdimgíµ ÷Æ°>©r6r)r ÚsignbitrrÚabsÚisnanÚisinfÚtensorrÚappendÚitemr%ÚviewÚshapeÚ unsqueezeÚmaxÚminÚsumÚexpandÚint32Úargmin)rÚvaluesrÚ abs_valuesr+Ú is_invalidÚE_bitsÚM_bitsÚcandidate_valuesÚ candidate_EÚ candidate_Mrr1r Ú significandr*Ú candidatesÚabs_values_flatÚNÚabs_values_expandedÚmax_candidate_valueÚerrorsÚ min_errorsÚ_Úis_tieÚM_bits_expandedÚ tie_breakerÚ best_indicesÚ E_selectedÚ M_selectedrrrr Nsd     û      û        zMXFP4Tensor._from_floatc Csî|j}d|kr|jksJdƒ‚Jdƒ‚| |¡}|dd}|ddkrIdgd|j}|j|ddd}d||<tjjj||ddd}t|jƒ}|||<|  |dd¡|j |Ž}|  |dd¡}|  |dd¡} | d>|B} | S)a Packs two e2m1 elements into a single uint8 along the specified dimension. Parameters: - dim: The dimension along which to pack the elements. Returns: - A torch tensor of dtype uint8 with two e2m1 elements packed into one uint8. rzHThe dimension to pack along is not within the range of tensor dimensionsrrÚconstant)Úmoder*r) r Úndimrr ÚnnÚ functionalÚpadÚlistrAÚinsertÚreshapeÚselect) rr6r Úsize_along_dimÚnew_size_along_dimÚ pad_sizesÚ pad_indexÚ new_shapeÚlowÚhighÚpackedrrrÚto_packed_tensor¦s* ÿÿ      zMXFP4Tensor.to_packed_tensorc Cs²|d?d@}|d@}tj||f|dd}t|jƒ}|d|…||dg||dd…}|j|Ž} ||ddkrStdƒg| j} td||ƒ| |<| t| ƒ} |  tj ¡S)aÅ Unpacks a tensor where two fp4 elements are packed into a single uint8. Parameters: - packed_tensor: The packed tensor - dim: The dimension along which the tensor was packed. - original_shape: The shape of the original tensor before packing. Returns: - A tensor with the original data unpacked into uint8 elements containing one fp4e2m1 element in the least significant bits. rérr8Nrr) r ÚstackrfrArhÚslicerbr rr) rÚ packed_tensorr6Úoriginal_shaperproÚstackedrArnr ÚindicesrrrÚunpack_packed_tensorÉs  *   z MXFP4Tensor.unpack_packed_tensor©NNN) Ú__name__Ú __module__Ú __qualname__rr!r4r rrrzrrrrr s #X #rc@s0eZdZd dd„Zd dd„Zdd„Zdd „ZdS) Ú MXScaleTensorNcCr)a6 Tensor class for working with microscaling E8M0 block scale factors. Parameters: - data: A torch tensor of float32 numbers to convert to fp8e8m0 microscaling format. - size: The size of the tensor to create. - device: The device on which to create the tensor. Nrrrrrrrrës  zMXScaleTensor.__init__c Cs–d}|durdntdtt t |¡¡ƒ|ƒ}|durdntdtdtt t |¡¡ƒ|ƒƒ}||ks7Jdƒ‚tj||d|jtj|j d}||_ |S)zp Generate random E8M0 data within a specified range. * Excludes the NaN encoding (255). éNréþz&Low must be less than or equal to highrr) rCÚintr Úlog2r=rDrrrrr )rrorpÚbiasÚ min_exponentÚ max_exponentrrrrr!þs*0zMXScaleTensor.randomcCs^|tjks Jdƒ‚|j |¡}|dk}| ¡}d||<|d}t d|¡}tj||<| |¡S)NzBCurrently only float32 is supported for f8e8m0 to float conversionéÿrr€g@)r r%r rÚcloner(Únan)rrr Úis_nanÚe_biasedÚer*rrrr4 s    zMXScaleTensor.toc CsŠtj|tj|jd}t |¡t |¡B|dkB}d||<||}t t |¡¡}|d}| tj ¡}t  |dd¡}| tj¡||<|S)aO Convert float32 numbers to E8M0 format. * Values <= 0, NaNs, and Infs are converted to the NaN encoding (255). * Positive values are converted by computing the floor of log2(value) to get the exponent. Parameters: - values: A torch tensor of float32 numbers to convert to E8M0 format. r5rr‡r€r) r Ú empty_likerrr;r<ÚfloorrƒrrGÚclamp) rrIÚresultrKÚ valid_valuesrŒr‹Ú e_biased_intÚe_biased_clampedrrrr s   zMXScaleTensor._from_floatr{)NN)r|r}r~rr!r4r rrrrrés    r)Ú__doc__r rrrrrrÚs ^