@[h\ddlZddlZddlmZejdZGddZdS)N) ProbingStates%[a-zA-Z]*[-]+[a-zA-Z]*[^a-zA-Z-]?ceZdZdZd dZdZedZdZedZ dZ e d Z e d Z e d ZdS) CharSetProbergffffff?Nc^d|_||_tjt|_dSN)_state lang_filterlogging getLogger__name__logger)selfr s p/var/www/api.easyaligner.net/htdocs/venv_linux/lib/python3.11/site-packages/pip/_vendor/chardet/charsetprober.py__init__zCharSetProber.__init__+s' &'11 c(tj|_dSr)r DETECTINGr rs rresetzCharSetProber.reset0s", rcdSrrs r charset_namezCharSetProber.charset_name3strctr)NotImplementedError)rbyte_strs rfeedzCharSetProber.feed7s!!rc|jSr)r rs rstatezCharSetProber.state:s {rcdS)Ngrrs rget_confidencezCharSetProber.get_confidence>ssrc2tjdd|}|S)Ns([-])+ )resub)bufs rfilter_high_byte_onlyz#CharSetProber.filter_high_byte_onlyAsf&c22 rct}t|}|D]Z}||dd|dd}|s|dkrd}||[|S)u7 We define three types of bytes: alphabet: english alphabets [a-zA-Z] international: international characters [€-ÿ] marker: everything else [^a-zA-Z€-ÿ] The input buffer can be thought to contain a series of words delimited by markers. This function works to filter all words that contain at least one international character. All contiguous sequences of markers are replaced by a single space ascii character. This filter applies to all scripts which do not use English characters. Nr#) bytearrayINTERNATIONAL_WORDS_PATTERNfindallextendisalpha)r&filteredwordsword last_chars rfilter_international_wordsz(CharSetProber.filter_international_wordsFs;; ,33C88 ' 'D OOD"I & & & RSS I$$&& !9w+>+> OOI & & & &rcvt}d}d}t|d}t|D]U\}}|dkr|dz}d}|dkr<||kr4|s2|||||dd}V|s|||d |S) a[ Returns a copy of ``buf`` that retains only the sequences of English alphabet and high byte characters that are not between <> characters. This filter can be applied to all scripts which contain both English characters and extended ASCII characters, but is currently only used by ``Latin1Prober``. Frc>rrHs~: (bj8 ccccccccccr