OwlCyberSecurity - MANAGER
Edit File: _inputstream.cpython-310.pyc
o ƚRea~ � @ s� d dl mZmZmZ d dlmZ d dlmZmZ d dl Z d dl Z d dl m Z m Z d dlmZ ddlmZmZmZmZ dd lmZ dd lmZ ed d � eD ��Zed d � eD ��Zedd � eD ��Zeeddg�B Z dZ ej r�e d dkr{e � d�dks}J �e � e dd� e!d� d �Z"ne � e �Z"h d�Z#e � d�Z$i Z%G dd� de&�Z'dd� Z(G dd � d e&�Z)G d d � d e)�Z*G d d � d e+�Z,G d!d"� d"e&�Z-G d#d$� d$e&�Z.d%d&� Z/dS )'� )�absolute_import�division�unicode_literals)� text_type)� http_client�urllibN)�BytesIO�StringIO)� webencodings� )�EOF�spaceCharacters� asciiLetters�asciiUppercase)�_ReparseException)�_utilsc C � g | ]}|� d ��qS ��ascii��encode��.0�item� r ��/builddir/build/BUILDROOT/alt-python310-pip-21.3.1-3.el8.x86_64/opt/alt/python310/lib/python3.10/site-packages/pip/_vendor/html5lib/_inputstream.py� <listcomp> � r c C r r r r r r r r r c C r r r r r r r r r � >� <u� [- - --]����]z"\uD800-\uDFFF"> i�� i�� i�� i�� i�� i�� i�� i�� i�� i�� i�� i�� i�� i�� i�� i�� i�� i�� i�� i�� i�� i�� i�� i�� i�� i�� i�� i�� i�� i�� i�� i�� z[ - -/:-@\[-`{-~]c @ sH e Zd ZdZdd� Zdd� Zdd� Zdd � Zd d � Zd d � Z dd� Z dS )�BufferedStreamz�Buffering for streams that do not have buffering of their own The buffer is implemented as a list of chunks on the assumption that joining many strings will be slow since it is O(n**2) c C s || _ g | _ddg| _d S )Nr r )�stream�buffer�position)�selfr# r r r �__init__: s zBufferedStream.__init__c C s< d}| j d | jd � D ]}|t|�7 }q || jd 7 }|S �Nr r )r$ r% �len)r& �pos�chunkr r r �tell? s zBufferedStream.tellc C sd || � � ksJ �|}d}t| j| �|k r+|t| j| �8 }|d7 }t| j| �|k s||g| _d S r( )�_bufferedBytesr) r$ r% )r& r* �offset�ir r r �seekF s �zBufferedStream.seekc C sP | j s| �|�S | jd t| j �kr#| jd t| j d �kr#| �|�S | �|�S )Nr r r )r$ � _readStreamr% r) �_readFromBuffer�r& �bytesr r r �readO s zBufferedStream.readc C s t dd� | jD ��S )Nc S s g | ]}t |��qS r )r) r r r r r Y s z1BufferedStream._bufferedBytes.<locals>.<listcomp>)�sumr$ �r& r r r r- X � z BufferedStream._bufferedBytesc C s< | j �|�}| j�|� | jd d7 < t|�| jd< |S r( )r# r5 r$ �appendr% r) )r& r4 �datar r r r1 [ s zBufferedStream._readStreamc C s� |}g }| j d }| j d }|t| j�k rc|dkrc|dks J �| j| }|t|�| kr6|}||| g| _ nt|�| }|t|�g| _ |d7 }|�|||| � � ||8 }d}|t| j�k rc|dks|rm|�| �|�� d�|�S )Nr r � )r% r) r$ r9 r1 �join)r& r4 �remainingBytes�rv� bufferIndex� bufferOffset� bufferedData� bytesToReadr r r r2 b s( � z BufferedStream._readFromBufferN) �__name__� __module__� __qualname__�__doc__r' r, r0 r5 r- r1 r2 r r r r r" 3 s r" c K s� t | tj�st | tjj�rt | jtj�rd}nt| d�r%t | �d�t �}nt | t �}|rCdd� |D �}|r;t d| ��t | fi |��S t | fi |��S )NFr5 r c S s g | ] }|� d �r|�qS )� _encoding)�endswith)r �xr r r r � s z#HTMLInputStream.<locals>.<listcomp>z3Cannot set an encoding with a unicode input, set %r) � isinstancer � HTTPResponser �response�addbase�fp�hasattrr5 r � TypeError�HTMLUnicodeInputStream�HTMLBinaryInputStream)�source�kwargs� isUnicode� encodingsr r r �HTMLInputStream} s � � rW c @ sp e Zd ZdZdZdd� Zdd� Zdd� Zd d � Zd d � Z d d� Z ddd�Z dd� Z dd� Z d dd�Zdd� ZdS ) rQ ��Provides a unicode stream of characters to the HTMLTokenizer. This class takes care of character encoding and removing or replacing incorrect byte-sequences and also provides column and line tracking. i ( c C sZ t jsd| _ntd�dkr| j| _n| j| _dg| _td�df| _| � |�| _ | � � dS )� Initialises the HTMLInputStream. HTMLInputStream(source, [encoding]) -> Normalized stream from source for use by html5lib. source can be either a file-object, local filename or a string. The optional encoding parameter must be a string that indicates the encoding. If specified, that encoding will be used, regardless of any BOM or later declaration (such as in a meta element) Nu r r �utf-8�certain) r �supports_lone_surrogates�reportCharacterErrorsr) �characterErrorsUCS4�characterErrorsUCS2�newLines�lookupEncoding� charEncoding� openStream� dataStream�reset)r& rS r r r r' � s z HTMLUnicodeInputStream.__init__c C s. d| _ d| _d| _g | _d| _d| _d | _d S )N� r )r+ � chunkSize� chunkOffset�errors� prevNumLines� prevNumCols�_bufferedCharacterr7 r r r re � s z HTMLUnicodeInputStream.resetc C s t |d�r |}|S t|�}|S �zvProduces a file object from source. source can be either a file object, local filename or a string. r5 )rO r �r& rS r# r r r rc � s �z!HTMLUnicodeInputStream.openStreamc C sZ | j }|�dd|�}| j| }|�dd|�}|dkr#| j| }||fS ||d }||fS )N� r r r )r+ �countrj �rfindrk )r& r. r+ �nLines� positionLine� lastLinePos�positionColumnr r r � _position� s �z HTMLUnicodeInputStream._positionc C s | � | j�\}}|d |fS )z:Returns (line, col) of the current position in the stream.r )rv rh )r& �line�colr r r r% � s z HTMLUnicodeInputStream.positionc C s6 | j | jkr | �� s tS | j }| j| }|d | _ |S )zo Read one character from the stream or queue if available. Return EOF when EOF is reached. r )rh rg � readChunkr r+ )r& rh �charr r r rz � s zHTMLUnicodeInputStream.charNc C s� |d u r| j }| �| j�\| _| _d| _d| _d| _| j�|�}| j r,| j | }d | _ n|s0dS t |�dkrWt |d �}|dksLd| krJdkrWn n |d | _ |d d� }| j r_| � |� |� d d �}|� d d �}|| _t |�| _d S ) Nrf r Fr r � � � i�� z ro � T)�_defaultChunkSizerv rg rj rk r+ rh rd r5 rl r) �ordr] �replace)r& rg r: �lastvr r r ry � s0 z HTMLUnicodeInputStream.readChunkc C s( t tt�|���D ]}| j�d� q d S )N�invalid-codepoint)�ranger) �invalid_unicode_re�findallri r9 )r& r: �_r r r r^ s �z*HTMLUnicodeInputStream.characterErrorsUCS4c C s� d}t �|�D ]Q}|r qt|�� �}|�� }t�|||d � �r9t�|||d � �}|tv r6| j � d� d}q|dkrP|dkrP|t |�d krP| j � d� qd}| j � d� qd S )NF� r� Tr| i�� r ) r� �finditerr �group�startr �isSurrogatePair�surrogatePairToCodepoint�non_bmp_invalid_codepointsri r9 r) )r& r: �skip�match� codepointr* �char_valr r r r_ # s" �z*HTMLUnicodeInputStream.characterErrorsUCS2Fc C s zt ||f }W n4 ty< |D ] }t|�dk s J �qd�dd� |D ��}|s-d| }t�d| � }t ||f< Y nw g } |�| j| j�}|du rT| j| j krSn*n|� � }|| j krl|� | j| j|� � || _n|� | j| jd� � | � � s|nq@d�|�} | S ) z� Returns a string of characters from the stream up to but not including any character in 'characters' or EOF. 'characters' must be a container that supports the 'in' method and iteration over its characters. T� rf c S s g | ]}d t |� �qS )z\x%02x)r )r �cr r r r H s z5HTMLUnicodeInputStream.charsUntil.<locals>.<listcomp>z^%sz[%s]+N) �charsUntilRegEx�KeyErrorr r< �re�compiler� r+ rh rg �endr9 ry ) r& � characters�opposite�charsr� �regexr> �mr� �rr r r � charsUntil: s: � � � z!HTMLUnicodeInputStream.charsUntilc C sZ |t ur)| jdkr|| j | _| jd7 _d S | jd8 _| j| j |ks+J �d S d S r( )r rh r+ rg )r& rz r r r �ungeti s � z HTMLUnicodeInputStream.unget�N)F)rC rD rE rF r~ r' re rc rv r% rz ry r^ r_ r� r� r r r r rQ � s & /rQ c @ sR e Zd ZdZ ddd�Zdd� Zd d � Zdd d �Zd d� Zdd� Z dd� Z dS )rR rX N� windows-1252Tc C sn | � |�| _t�| | j� d| _d| _|| _|| _|| _|| _ || _ | � |�| _ | j d dus1J �| � � dS )rY i �d r N)rc � rawStreamrQ r' � numBytesMeta�numBytesChardet�override_encoding�transport_encoding�same_origin_parent_encoding�likely_encoding�default_encoding�determineEncodingrb re )r& rS r� r� r� r� r� � useChardetr r r r' � s z HTMLBinaryInputStream.__init__c C s& | j d j�| jd�| _t�| � d S )Nr r� )rb � codec_info� streamreaderr� rd rQ re r7 r r r re � s zHTMLBinaryInputStream.resetc C sJ t |d�r|}nt|�}z |�|�� � W |S ty$ t|�}Y |S w rm )rO r r0 r, � Exceptionr" rn r r r rc � s � �z HTMLBinaryInputStream.openStreamc C s� | � � df}|d d ur|S t| j�df}|d d ur |S t| j�df}|d d ur,|S | �� df}|d d ur:|S t| j�df}|d d urQ|d j�d�sQ|S t| j�df}|d d ur`|S |r�zddl m } W n t ys Y n@w g }|� }|j s�| j �| j�}t|t�s�J �|s�n |�|� |�|� |j r||�� t|jd �}| j �d� |d ur�|dfS t| j�df}|d d ur�|S td�dfS )Nr[ r � tentativezutf-16)�UniversalDetector�encodingr� )� detectBOMra r� r� �detectEncodingMetar� �name� startswithr� �%pip._vendor.chardet.universaldetectorr� � ImportError�doner� r5 r� rJ r4 r9 �feed�close�resultr0 r� )r& �chardetrb r� �buffers�detectorr$ r� r r r r� � sV � � z'HTMLBinaryInputStream.determineEncodingc C s� | j d dks J �t|�}|d u rd S |jdv r$td�}|d us"J �d S || j d kr5| j d df| _ d S | j�d� |df| _ | �� td| j d |f ��)Nr r[ ��utf-16be�utf-16lerZ r z Encoding changed from %s to %s)rb ra r� r� r0 re r )r& � newEncodingr r r �changeEncoding s z$HTMLBinaryInputStream.changeEncodingc C s� t jdt jdt jdt jdt jdi}| j�d�}t|t �s J �|� |dd� �}d}|s?|� |�}d}|s?|� |dd � �}d }|rK| j� |� t |�S | j� d � dS ) z�Attempts to detect at BOM at the start of the stream. If an encoding can be determined from the BOM return the name of the encoding otherwise return NonerZ r� r� zutf-32lezutf-32be� N� r� r ) �codecs�BOM_UTF8� BOM_UTF16_LE� BOM_UTF16_BE� BOM_UTF32_LE� BOM_UTF32_BEr� r5 rJ r4 �getr0 ra )r& �bomDict�stringr� r0 r r r r� s& � z HTMLBinaryInputStream.detectBOMc C sV | j �| j�}t|t�sJ �t|�}| j �d� |�� }|dur)|jdv r)t d�}|S )z9Report the encoding declared by the meta element r Nr� rZ ) r� r5 r� rJ r4 �EncodingParserr0 � getEncodingr� ra )r& r$ �parserr� r r r r� 3 s z(HTMLBinaryInputStream.detectEncodingMeta)NNNNr� T)T) rC rD rE rF r' re rc r� r� r� r� r r r r rR z s �* > "rR c @ s� e Zd ZdZdd� Zdd� Zdd� Zdd � Zd d � Zd d � Z dd� Z dd� Z e e e �Z dd� Ze e�Zefdd�Zdd� Zdd� Zdd� Zd S ) � EncodingBytesz�String-like object with an associated position and various extra methods If the position is ever greater than the string length then an exception is raisedc C s t |t�sJ �t�| |�� �S r� )rJ r4 �__new__�lower�r& �valuer r r r� F s zEncodingBytes.__new__c C s d| _ d S )Nr )rv r� r r r r' J s zEncodingBytes.__init__c C s | S r� r r7 r r r �__iter__N s zEncodingBytes.__iter__c C s< | j d }| _ |t| �krt�|dk rt�| ||d � S )Nr r �rv r) � StopIterationrP �r& �pr r r �__next__Q s zEncodingBytes.__next__c C s | � � S r� )r� r7 r r r �nextY s zEncodingBytes.nextc C s@ | j }|t| �kr t�|dk rt�|d | _ }| ||d � S r( r� r� r r r �previous] s zEncodingBytes.previousc C s | j t| �kr t�|| _ d S r� �rv r) r� )r& r% r r r � setPositionf s zEncodingBytes.setPositionc C s& | j t| �kr t�| j dkr| j S d S )Nr r� r7 r r r � getPositionk s zEncodingBytes.getPositionc C s | | j | j d � S �Nr )r% r7 r r r �getCurrentByteu r8 z EncodingBytes.getCurrentBytec C sR | j }|t| �k r$| ||d � }||vr|| _|S |d7 }|t| �k s || _dS )z Skip past a list of charactersr N�r% r) rv �r& r� r� r� r r r r� z s �zEncodingBytes.skipc C sR | j }|t| �k r$| ||d � }||v r|| _|S |d7 }|t| �k s || _d S r� r� r� r r r � skipUntil� s �zEncodingBytes.skipUntilc C s( | � || j�}|r| jt|�7 _|S )z�Look for a sequence of bytes at the start of a string. If the bytes are found return True and advance the position to the byte after the match. Otherwise return False and leave the position alone)r� r% r) )r& r4 r> r r r � matchBytes� s zEncodingBytes.matchBytesc C s6 z| � || j�t|� d | _W dS ty t�w )z�Look for the next sequence of bytes matching a given sequence. If a match is found advance the position to the last byte of the matchr T)�indexr% r) rv � ValueErrorr� r3 r r r �jumpTo� s ��zEncodingBytes.jumpToN)rC rD rE rF r� r' r� r� r� r� r� r� �propertyr% r� � currentByte�spaceCharactersBytesr� r� r� r� r r r r r� B s" r� c @ sX e Zd ZdZdd� Zdd� Zdd� Zdd � Zd d � Zd d � Z dd� Z dd� Z dd� Z dS )r� z?Mini parser for detecting character encoding from meta elementsc C s t |�| _d| _dS )z3string - the data to work on for encoding detectionN)r� r: r� �r& r: r r r r' � s zEncodingParser.__init__c C s� d| j vrd S d| jfd| jfd| jfd| jfd| jfd| jff}| j D ]?}d}z| j �d� W n ty< Y | j S w |D ] \}}| j �|�r\z|� }W n ty[ d}Y nw q?|sc | j S q$| j S ) Ns <metas <!--s </s <!s <?r TF) r: � handleComment� handleMeta�handlePossibleEndTag� handleOther�handlePossibleStartTagr� r� r� r� )r& �methodDispatchr� � keepParsing�key�methodr r r r� � s@ � � ���zEncodingParser.getEncodingc C � | j �d�S )zSkip over commentss -->�r: r� r7 r r r r� � s z EncodingParser.handleCommentc C s� | j jtvrdS d}d } | �� }|d u rdS |d dkr/|d dk}|r.|d ur.|| _dS n?|d dkrG|d }t|�}|d urF|| _dS n'|d dkrntt|d ��}|�� }|d urnt|�}|d urn|rl|| _dS |}q ) NTFr s http-equivr s content-type� charsets content) r: r� r� � getAttributer� ra �ContentAttrParserr� �parse)r& � hasPragma�pendingEncoding�attr�tentativeEncoding�codec� contentParserr r r r� � s@ � � �zEncodingParser.handleMetac C s | � d�S )NF)�handlePossibleTagr7 r r r r� � s z%EncodingParser.handlePossibleStartTagc C s t | j� | �d�S )NT)r� r: r r7 r r r r� � s z#EncodingParser.handlePossibleEndTagc C sj | j }|jtvr|r|�� | �� dS |�t�}|dkr#|�� dS | �� }|d ur3| �� }|d us+dS )NTr )r: r� �asciiLettersBytesr� r� r� �spacesAngleBracketsr� )r& �endTagr: r� r r r r r � s ��z EncodingParser.handlePossibleTagc C r� )Nr r� r7 r r r r� s zEncodingParser.handleOtherc C s� | j }|�ttdg�B �}|du st|�dksJ �|dv r dS g }g } |dkr+|r+n0|tv r4|�� }n'|dv r?d�|�dfS |tv rK|�|�� � n |du rQdS |�|� t |�}q$|dkrj|� � d�|�dfS t |� |�� }|d v r�|} t |�}||kr�t |� d�|�d�|�fS |tv r�|�|�� � n|�|� qy|d kr�d�|�dfS |tv r�|�|�� � n |du r�dS |�|� t |�}|t v r�d�|�d�|�fS |tv r�|�|�� � n |du r�dS |�|� q�) z_Return a name,value pair for the next attribute in the stream, if one is found, or None� /Nr )r NT� =)r r r; )� '� "r ) r: r� r� � frozensetr) r< �asciiUppercaseBytesr9 r� r� r� r )r& r: r� �attrName� attrValue� quoteCharr r r r� sn � � �zEncodingParser.getAttributeN) rC rD rE rF r' r� r� r� r� r� r r� r� r r r r r� � s $ r� c @ s e Zd Zdd� Zdd� ZdS )r� c C s t |t�sJ �|| _d S r� )rJ r4 r: r� r r r r' a s zContentAttrParser.__init__c C s zy| j �d� | j jd7 _| j �� | j jdks W d S | j jd7 _| j �� | j jdv rS| j j}| j jd7 _| j j}| j �|�rP| j || j j� W S W d S | j j}z| j �t� | j || j j� W W S tyy | j |d � Y W S w ty� Y d S w )Nr� r r )r r )r: r� r% r� r� r� r� r� )r& � quoteMark� oldPositionr r r r� e s2 � �zContentAttrParser.parseN)rC rD rE r' r� r r r r r� ` s r� c C s\ t | t�rz| �d�} W n ty Y dS w | dur,zt�| �W S ty+ Y dS w dS )z{Return the python codec name corresponding to an encoding or None if the string doesn't correspond to a valid encoding.r N)rJ r4 �decode�UnicodeDecodeErrorr �lookup�AttributeError)r� r r r ra � s � �ra )0� __future__r r r Zpip._vendor.sixr �pip._vendor.six.movesr r r� r� �ior r � pip._vendorr � constantsr r r r r rf r r r� r r r � invalid_unicode_no_surrogater\ rp r� �evalr� r� �ascii_punctuation_rer� �objectr" rW rQ rR r4 r� r� r� ra r r r r �<module> sJ �� J g Ib = '