"codecs" — Registre des codecs et classes de base associées
***********************************************************

**Code source :** Lib/codecs.py

======================================================================

Ce module définit les classes de base pour les codecs (encodeurs et
décodeurs) standards Python et fournit l'interface avec le registre
des codecs internes à Python, qui gère le processus de recherche de
codecs et de gestion des erreurs. La plupart des codecs sont des
*encodeurs de texte*, qui encode du texte vers des séquences d'octets
(type *bytes* de Python) mais il existe aussi des codecs qui encodent
du texte vers du texte et des *bytes* vers des *bytes*. Les codecs
personnalisés peuvent encoder et décoder des types arbitraires, mais
l'utilisation de certaines fonctionnalités du module est restreinte
aux *encodeurs de texte* ou aux codecs qui encodent vers "bytes".

Le module définit les fonctions suivantes pour encoder et décoder à
l'aide de n'importe quel codec :

codecs.encode(obj, encoding='utf-8', errors='strict')

   Encode *obj* en utilisant le codec enregistré pour *encoding*.

   Vous pouvez spécifier *errors* pour définir la façon de gérer les
   erreurs. Le gestionnaire d'erreurs par défaut est "'strict'", ce
   qui veut dire qu'une erreur lors de l'encodage lève "ValueError"
   (ou une sous-classe spécifique du codec, telle que
   "UnicodeEncodeError"). Référez-vous aux Classes de base de codecs
   pour plus d'informations sur la gestion des erreurs par les codecs.

codecs.decode(obj, encoding='utf-8', errors='strict')

   Décode *obj* en utilisant le codec enregistré pour *encoding*.

   Vous pouvez spécifier *errors* pour définir la façon de gérer les
   erreurs. Le gestionnaire d'erreurs par défaut est "'strict'", ce
   qui veut dire qu'une erreur lors du décodage lève "ValueError" (ou
   une sous-classe spécifique du codec, telle que
   "UnicodeDecodeError"). Référez-vous aux Classes de base de codecs
   pour plus d'informations sur la gestion des erreurs par les codecs.

Les détails complets de chaque codec peuvent être examinés directement
:

codecs.lookup(encoding)

   Recherche les informations relatives au codec dans le registre des
   codecs de Python et renvoie l'objet "CodecInfo" tel que défini ci-
   dessous.

   Les encodeurs sont recherchés en priorité dans le cache du
   registre. S'ils n'y sont pas, la liste des fonctions de recherche
   enregistrées est passée en revue. Si aucun objet "CodecInfo" n'est
   trouvé, une "LookupError" est levée. Sinon, l'objet "CodecInfo" est
   mis en cache et renvoyé vers l'appelant.

class codecs.CodecInfo(encode, decode, streamreader=None, streamwriter=None, incrementalencoder=None, incrementaldecoder=None, name=None)

   Les détails d'un codec trouvé dans le registre des codecs. Les
   arguments du constructeur sont stockés dans les attributs éponymes
   :

   name

      Le nom de l'encodeur.

   encode
   decode

      Les fonctions d'encodage et de décodage. Ces fonctions ou
      méthodes doivent avoir la même interface que les méthodes
      "encode()" et "decode()" des instances de Codec (voir Interface
      des codecs). Les fonctions et méthodes sont censées fonctionner
      sans état interne.

   incrementalencoder
   incrementaldecoder

      Classes d'encodeurs et de décodeurs incrémentaux ou fonctions
      usines. Elles doivent avoir respectivement les mêmes interfaces
      que celles définies par les classes de base "IncrementalEncoder"
      et "IncrementalDecoder". Les codecs incrémentaux peuvent
      conserver des états internes.

   streamwriter
   streamreader

      Classes d'écriture et de lecture de flux ou fonctions usines.
      Elles doivent avoir les mêmes interfaces que celles définies par
      les classes de base "StreamWriter" et "StreamReader",
      respectivement. Les codecs de flux peuvent conserver un état
      interne.

Pour simplifier l'accès aux différents composants du codec, le module
fournit les fonctions supplémentaires suivantes qui utilisent
"lookup()" pour la recherche du codec :

codecs.getencoder(encoding)

   Recherche le codec pour l'encodage *encoding* et renvoie sa
   fonction d'encodage.

   Lève une "LookupError" si l'encodage *encoding* n'est pas trouvé.

codecs.getdecoder(encoding)

   Recherche le codec pour l'encodage *encoding* et renvoie sa
   fonction de décodage.

   Lève une "LookupError" si l'encodage *encoding* n'est pas trouvé.

codecs.getincrementalencoder(encoding)

   Recherche le codec pour l'encodage *encoding* et renvoie sa classe
   d'encodage incrémental ou la fonction usine.

   Lève une "LookupError" si l'encodage *encoding* n'est pas trouvé ou
   si le codec ne gère pas l'encodage incrémental.

codecs.getincrementaldecoder(encoding)

   Recherche le codec pour l'encodage *encoding* et renvoie sa classe
   de décodage incrémental ou la fonction usine.

   Lève une "LookupError" si l'encodage *encoding* n'est pas trouvé ou
   si le codec ne gère pas le décodage incrémental.

codecs.getreader(encoding)

   Recherche le codec pour l'encodage *encoding* et renvoie sa classe
   "StreamReader" ou la fonction usine.

   Lève une "LookupError" si l'encodage *encoding* n'est pas trouvé.

codecs.getwriter(encoding)

   Recherche le codec pour l'encodage *encoding* et renvoie sa classe
   "StreamWriter" ou la fonction usine.

   Lève une "LookupError" si l'encodage *encoding* n'est pas trouvé.

Les codecs personnalisés sont mis à disposition en enregistrant une
fonction de recherche de codecs adaptée :

codecs.register(search_function)

   Enregistre une fonction de recherche de codec. Il convient qu'une
   fonction de recherche prenne un argument, le nom de l'encodage
   écrit en lettres minuscules, et renvoie un objet "CodecInfo". Si la
   fonction de recherche ne trouve pas un encodage donné, il convient
   qu'elle renvoie `` None``.

   Note:

     l'enregistrement d'une fonction de recherche n'est actuellement
     pas réversible, ce qui peut entraîner des problèmes dans certains
     cas, par exemple pour les tests unitaires ou le rechargement de
     module.

Alors qu'il est recommandé d'utiliser la fonction native "open()" et
le module associé "io" pour travailler avec des fichiers texte
encodés, le présent module fournit des fonctions et classes
utilitaires supplémentaires qui permettent l'utilisation d'une plus
large gamme de codecs si vous travaillez avec des fichiers binaires :

codecs.open(filename, mode='r', encoding=None, errors='strict', buffering=-1)

   Ouvre un fichier encodé en utilisant le *mode* donné et renvoie une
   instance de "StreamReaderWriter", permettant un encodage-décodage
   transparent. Le mode de fichier par défaut est "'r'", ce qui
   signifie que le fichier est ouvert en lecture.

   Note:

     les fichiers encodés sous-jacents sont toujours ouverts en mode
     binaire. Aucune conversion automatique de "'\n'" n'est effectuée
     à la lecture ou à l'écriture. L'argument *mode* peut être
     n'importe quel mode binaire acceptable pour la fonction native
     "open()" ; le "'b'" est automatiquement ajouté.

   *encoding* spécifie l'encodage à utiliser pour le fichier. Tout
   encodage qui encode et décode des octets (type *bytes*) est
   autorisé et les types de données pris en charge par les méthodes
   relatives aux fichiers dépendent du codec utilisé.

   *errors* peut être spécifié pour définir la gestion des erreurs. La
   valeur par défaut est "'strict'", ce qui lève une "ValueError" en
   cas d'erreur lors du codage.

   *buffering* a la même signification que pour la fonction native
   "open()". Il vaut *-1* par défaut, ce qui signifie que la taille
   par défaut du tampon est utilisée.

codecs.EncodedFile(file, data_encoding, file_encoding=None, errors='strict')

   Renvoie une instance de "StreamRecoder", version encapsulée de
   *file* qui fournit un transcodage transparent. Le fichier original
   est fermé quand la version encapsulée est fermée.

   Les données écrites dans un fichier encapsulant sont décodées en
   fonction du *data_encoding* spécifié puis écrites vers le fichier
   original en tant que *bytes* en utilisant *file_encoding*. Les
   octets lus dans le fichier original sont décodés conformément à
   *file_encoding* et le résultat est encodé en utilisant
   *data_encoding*.

   Si *file_encoding* n'est pas spécifié, la valeur par défaut est
   *data_encoding*.

   *errors* peut être spécifié pour définir la gestion des erreurs. La
   valeur par défaut est "'strict'", ce qui lève une "ValueError" en
   cas d'erreur lors du codage.

codecs.iterencode(iterator, encoding, errors='strict', **kwargs)

   Utilise un encodeur incrémental pour encoder de manière itérative
   l'entrée fournie par *iterator*. Cette fonction est un
   *générateur*. L'argument *errors* (ainsi que tout autre argument
   passé par mot-clé) est transmis à l'encodeur incrémental.

   Cette fonction nécessite que le codec accepte les objets texte
   (classe "str") en entrée. Par conséquent, il ne prend pas en charge
   les encodeurs *bytes* vers *bytes* tels que "base64_codec".

codecs.iterdecode(iterator, encoding, errors='strict', **kwargs)

   Utilise un décodeur incrémental pour décoder de manière itérative
   l'entrée fournie par *iterator*. Cette fonction est un
   *générateur*. L'argument *errors* (ainsi que tout autre argument
   passé par mot-clé) est transmis au décodeur incrémental.

   Cette fonction requiert que le codec accepte les objets "bytes" en
   entrée. Par conséquent, elle ne prend pas en charge les encodeurs
   de texte vers texte tels que "rot_13", bien que "rot_13" puisse
   être utilisé de manière équivalente avec "iterencode()".

Le module fournit également les constantes suivantes qui sont utiles
pour lire et écrire les fichiers dépendants de la plateforme :

codecs.BOM
codecs.BOM_BE
codecs.BOM_LE
codecs.BOM_UTF8
codecs.BOM_UTF16
codecs.BOM_UTF16_BE
codecs.BOM_UTF16_LE
codecs.BOM_UTF32
codecs.BOM_UTF32_BE
codecs.BOM_UTF32_LE

   Ces constantes définissent diverses séquences d'octets, les marques
   d'ordre d'octets (BOM pour *byte order mark* en anglais) Unicode
   pour plusieurs encodages. Elles sont utilisées dans les flux de
   données UTF-16 et UTF-32 pour indiquer l'ordre des octets utilisé,
   et dans UTF-8 comme signature Unicode. "BOM_UTF16" vaut soit
   "BOM_UTF16_BE", soit "BOM_UTF16_LE" selon le boutisme natif de la
   plateforme, "BOM" est un alias pour "BOM_UTF16", "BOM_LE" pour
   "BOM_UTF16_LE" et "BOM_BE" pour "BOM_UTF16_BE". Les autres sont les
   marques BOM dans les encodages UTF-8 et UTF-32.


Classes de base de codecs
=========================

Le module "codecs" définit un ensemble de classes de base qui
spécifient les interfaces pour travailler avec des objets codecs et
qui peuvent également être utilisées comme base pour des
implémentations de codecs personnalisés.

Chaque codec doit définir quatre interfaces pour être utilisable comme
codec en Python : codeur sans état, décodeur sans état, lecteur de
flux et écrivain de flux. Le lecteur et l'écrivain de flux réutilisent
généralement l'encodeur-décodeur sans état pour implémenter les
protocoles de fichiers. Les auteurs de codecs doivent également
définir comment le codec gère les erreurs d'encodage et de décodage.


Gestionnaires d'erreurs
-----------------------

Pour simplifier et standardiser la gestion des erreurs, les codecs
peuvent implémenter différents schémas de gestion des erreurs en
acceptant l'argument *errors* de type chaîne. Les chaînes suivantes
sont définies et les gestionnaires correspondants implémentés par tous
les codecs Python standards :

+---------------------------+-------------------------------------------------+
| Valeur                    | Signification                                   |
|===========================|=================================================|
| "'strict'"                | Lève "UnicodeError" (ou une sous-classe) ;      |
|                           | c'est la valeur par défaut. Implémenté dans     |
|                           | "strict_errors()".                              |
+---------------------------+-------------------------------------------------+
| "'ignore'"                | Ignore les données incorrectement formatées et  |
|                           | continue sans rien signaler. Implémenté dans    |
|                           | "ignore_errors()".                              |
+---------------------------+-------------------------------------------------+

Les gestionnaires d'erreurs suivants ne s'appliquent que pour les
*encodeurs de texte* :

+---------------------------+-------------------------------------------------+
| Valeur                    | Signification                                   |
|===========================|=================================================|
| "'replace'"               | Remplace avec une marque de substitution        |
|                           | adaptée ; Python utilise le caractère de        |
|                           | substitution officiel "U+FFFD" pour les codecs  |
|                           | natifs lors du décodage et *'?'* lors de        |
|                           | l'encodage. Implémenté dans "replace_errors()". |
+---------------------------+-------------------------------------------------+
| "'xmlcharrefreplace'"     | Remplace avec la référence de caractère XML     |
|                           | adaptée (uniquement pour l'encodage).           |
|                           | Implémenté dans "xmlcharrefreplace_errors()".   |
+---------------------------+-------------------------------------------------+
| "'backslashreplace'"      | Remplace avec une séquence échappée par des     |
|                           | antislashs. Implémenté dans                     |
|                           | "backslashreplace_errors()".                    |
+---------------------------+-------------------------------------------------+
| "'namereplace'"           | Remplace avec des séquences échappées par       |
|                           | "\N{...}" (uniquement pour l'encodage).         |
|                           | Implémenté dans "namereplace_errors()".         |
+---------------------------+-------------------------------------------------+
| "'surrogateescape'"       | Lors du décodage, remplace un octet par un code |
|                           | de substitution individuel allant de "U+DC80" à |
|                           | "U+DCFF". Ce code est reconverti vers l'octet   |
|                           | de départ quand le gestionnaire d'erreurs       |
|                           | "'surrogateescape'" est utilisé pour l'encodage |
|                           | des données (voir la **PEP 383** pour plus de   |
|                           | détails).                                       |
+---------------------------+-------------------------------------------------+

En plus, le gestionnaire d'erreurs suivant est spécifique aux codecs
suivants :

+---------------------+--------------------------+---------------------------------------------+
| Valeur              | Codecs                   | Signification                               |
|=====================|==========================|=============================================|
| "'surrogatepass'"   | utf-8, utf-16, utf-32,   | Autorise l'encodage et le décodage de codes |
|                     | utf-16-be, utf-16-le,    | de substitution. Ces codecs traitent        |
|                     | utf-32-be, utf-32-le     | normalement la présence d'un code de        |
|                     |                          | substitution comme une erreur.              |
+---------------------+--------------------------+---------------------------------------------+

Nouveau dans la version 3.1: les gestionnaires d'erreurs
"'surrogateescape'" et "'surrogatepass'".

Modifié dans la version 3.4: le gestionnaire d'erreurs
"'surrogatepass'" fonctionne maintenant avec les codecs utf-16* et
utf-32*.

Nouveau dans la version 3.5: le gestionnaire d'erreurs
"'namereplace'".

Modifié dans la version 3.5: le gestionnaire d'erreurs
"'backslashreplace'" fonctionne maintenant pour le décodage et la
traduction.

L'ensemble des valeurs autorisées peut être étendu en enregistrant un
nouveau gestionnaire d'erreurs nommé :

codecs.register_error(name, error_handler)

   Register the error handling function *error_handler* under the name
   *name*. The *error_handler* argument will be called during encoding
   and decoding in case of an error, when *name* is specified as the
   errors parameter.

   For encoding, *error_handler* will be called with a
   "UnicodeEncodeError" instance, which contains information about the
   location of the error. The error handler must either raise this or
   a different exception, or return a tuple with a replacement for the
   unencodable part of the input and a position where encoding should
   continue. The replacement may be either "str" or "bytes". If the
   replacement is bytes, the encoder will simply copy them into the
   output buffer. If the replacement is a string, the encoder will
   encode the replacement. Encoding continues on original input at the
   specified position. Negative position values will be treated as
   being relative to the end of the input string. If the resulting
   position is out of bound an "IndexError" will be raised.

   Decoding and translating works similarly, except
   "UnicodeDecodeError" or "UnicodeTranslateError" will be passed to
   the handler and that the replacement from the error handler will be
   put into the output directly.

Previously registered error handlers (including the standard error
handlers) can be looked up by name:

codecs.lookup_error(name)

   Return the error handler previously registered under the name
   *name*.

   Raises a "LookupError" in case the handler cannot be found.

The following standard error handlers are also made available as
module level functions:

codecs.strict_errors(exception)

   Implements the "'strict'" error handling: each encoding or decoding
   error raises a "UnicodeError".

codecs.replace_errors(exception)

   Implements the "'replace'" error handling (for *text encodings*
   only): substitutes "'?'" for encoding errors (to be encoded by the
   codec), and "'\ufffd'" (the Unicode replacement character) for
   decoding errors.

codecs.ignore_errors(exception)

   Implements the "'ignore'" error handling: malformed data is ignored
   and encoding or decoding is continued without further notice.

codecs.xmlcharrefreplace_errors(exception)

   Implements the "'xmlcharrefreplace'" error handling (for encoding
   with *text encodings* only): the unencodable character is replaced
   by an appropriate XML character reference.

codecs.backslashreplace_errors(exception)

   Implements the "'backslashreplace'" error handling (for *text
   encodings* only): malformed data is replaced by a backslashed
   escape sequence.

codecs.namereplace_errors(exception)

   Implements the "'namereplace'" error handling (for encoding with
   *text encodings* only): the unencodable character is replaced by a
   "\N{...}" escape sequence.

   Nouveau dans la version 3.5.


Stateless Encoding and Decoding
-------------------------------

The base "Codec" class defines these methods which also define the
function interfaces of the stateless encoder and decoder:

Codec.encode(input[, errors])

   Encodes the object *input* and returns a tuple (output object,
   length consumed). For instance, *text encoding* converts a string
   object to a bytes object using a particular character set encoding
   (e.g., "cp1252" or "iso-8859-1").

   The *errors* argument defines the error handling to apply. It
   defaults to "'strict'" handling.

   The method may not store state in the "Codec" instance. Use
   "StreamWriter" for codecs which have to keep state in order to make
   encoding efficient.

   The encoder must be able to handle zero length input and return an
   empty object of the output object type in this situation.

Codec.decode(input[, errors])

   Decodes the object *input* and returns a tuple (output object,
   length consumed). For instance, for a *text encoding*, decoding
   converts a bytes object encoded using a particular character set
   encoding to a string object.

   For text encodings and bytes-to-bytes codecs, *input* must be a
   bytes object or one which provides the read-only buffer interface
   -- for example, buffer objects and memory mapped files.

   The *errors* argument defines the error handling to apply. It
   defaults to "'strict'" handling.

   The method may not store state in the "Codec" instance. Use
   "StreamReader" for codecs which have to keep state in order to make
   decoding efficient.

   The decoder must be able to handle zero length input and return an
   empty object of the output object type in this situation.


Incremental Encoding and Decoding
---------------------------------

The "IncrementalEncoder" and "IncrementalDecoder" classes provide the
basic interface for incremental encoding and decoding.
Encoding/decoding the input isn't done with one call to the stateless
encoder/decoder function, but with multiple calls to the
"encode()"/"decode()" method of the incremental encoder/decoder. The
incremental encoder/decoder keeps track of the encoding/decoding
process during method calls.

The joined output of calls to the "encode()"/"decode()" method is the
same as if all the single inputs were joined into one, and this input
was encoded/decoded with the stateless encoder/decoder.


IncrementalEncoder Objects
~~~~~~~~~~~~~~~~~~~~~~~~~~

The "IncrementalEncoder" class is used for encoding an input in
multiple steps. It defines the following methods which every
incremental encoder must define in order to be compatible with the
Python codec registry.

class codecs.IncrementalEncoder(errors='strict')

   Constructor for an "IncrementalEncoder" instance.

   All incremental encoders must provide this constructor interface.
   They are free to add additional keyword arguments, but only the
   ones defined here are used by the Python codec registry.

   The "IncrementalEncoder" may implement different error handling
   schemes by providing the *errors* keyword argument. See
   Gestionnaires d'erreurs for possible values.

   The *errors* argument will be assigned to an attribute of the same
   name. Assigning to this attribute makes it possible to switch
   between different error handling strategies during the lifetime of
   the "IncrementalEncoder" object.

   encode(object[, final])

      Encodes *object* (taking the current state of the encoder into
      account) and returns the resulting encoded object. If this is
      the last call to "encode()" *final* must be true (the default is
      false).

   reset()

      Reset the encoder to the initial state. The output is discarded:
      call ".encode(object, final=True)", passing an empty byte or
      text string if necessary, to reset the encoder and to get the
      output.

   getstate()

      Return the current state of the encoder which must be an
      integer. The implementation should make sure that "0" is the
      most common state. (States that are more complicated than
      integers can be converted into an integer by marshaling/pickling
      the state and encoding the bytes of the resulting string into an
      integer.)

   setstate(state)

      Set the state of the encoder to *state*. *state* must be an
      encoder state returned by "getstate()".


IncrementalDecoder Objects
~~~~~~~~~~~~~~~~~~~~~~~~~~

The "IncrementalDecoder" class is used for decoding an input in
multiple steps. It defines the following methods which every
incremental decoder must define in order to be compatible with the
Python codec registry.

class codecs.IncrementalDecoder(errors='strict')

   Constructor for an "IncrementalDecoder" instance.

   All incremental decoders must provide this constructor interface.
   They are free to add additional keyword arguments, but only the
   ones defined here are used by the Python codec registry.

   The "IncrementalDecoder" may implement different error handling
   schemes by providing the *errors* keyword argument. See
   Gestionnaires d'erreurs for possible values.

   The *errors* argument will be assigned to an attribute of the same
   name. Assigning to this attribute makes it possible to switch
   between different error handling strategies during the lifetime of
   the "IncrementalDecoder" object.

   decode(object[, final])

      Decodes *object* (taking the current state of the decoder into
      account) and returns the resulting decoded object. If this is
      the last call to "decode()" *final* must be true (the default is
      false). If *final* is true the decoder must decode the input
      completely and must flush all buffers. If this isn't possible
      (e.g. because of incomplete byte sequences at the end of the
      input) it must initiate error handling just like in the
      stateless case (which might raise an exception).

   reset()

      Reset the decoder to the initial state.

   getstate()

      Return the current state of the decoder. This must be a tuple
      with two items, the first must be the buffer containing the
      still undecoded input. The second must be an integer and can be
      additional state info. (The implementation should make sure that
      "0" is the most common additional state info.) If this
      additional state info is "0" it must be possible to set the
      decoder to the state which has no input buffered and "0" as the
      additional state info, so that feeding the previously buffered
      input to the decoder returns it to the previous state without
      producing any output. (Additional state info that is more
      complicated than integers can be converted into an integer by
      marshaling/pickling the info and encoding the bytes of the
      resulting string into an integer.)

   setstate(state)

      Set the state of the decoder to *state*. *state* must be a
      decoder state returned by "getstate()".


Stream Encoding and Decoding
----------------------------

The "StreamWriter" and "StreamReader" classes provide generic working
interfaces which can be used to implement new encoding submodules very
easily. See "encodings.utf_8" for an example of how this is done.


StreamWriter Objects
~~~~~~~~~~~~~~~~~~~~

The "StreamWriter" class is a subclass of "Codec" and defines the
following methods which every stream writer must define in order to be
compatible with the Python codec registry.

class codecs.StreamWriter(stream, errors='strict')

   Constructor for a "StreamWriter" instance.

   All stream writers must provide this constructor interface. They
   are free to add additional keyword arguments, but only the ones
   defined here are used by the Python codec registry.

   The *stream* argument must be a file-like object open for writing
   text or binary data, as appropriate for the specific codec.

   The "StreamWriter" may implement different error handling schemes
   by providing the *errors* keyword argument. See Gestionnaires
   d'erreurs for the standard error handlers the underlying stream
   codec may support.

   The *errors* argument will be assigned to an attribute of the same
   name. Assigning to this attribute makes it possible to switch
   between different error handling strategies during the lifetime of
   the "StreamWriter" object.

   write(object)

      Writes the object's contents encoded to the stream.

   writelines(list)

      Writes the concatenated list of strings to the stream (possibly
      by reusing the "write()" method). The standard bytes-to-bytes
      codecs do not support this method.

   reset()

      Flushes and resets the codec buffers used for keeping state.

      Calling this method should ensure that the data on the output is
      put into a clean state that allows appending of new fresh data
      without having to rescan the whole stream to recover state.

In addition to the above methods, the "StreamWriter" must also inherit
all other methods and attributes from the underlying stream.


StreamReader Objects
~~~~~~~~~~~~~~~~~~~~

The "StreamReader" class is a subclass of "Codec" and defines the
following methods which every stream reader must define in order to be
compatible with the Python codec registry.

class codecs.StreamReader(stream, errors='strict')

   Constructor for a "StreamReader" instance.

   All stream readers must provide this constructor interface. They
   are free to add additional keyword arguments, but only the ones
   defined here are used by the Python codec registry.

   The *stream* argument must be a file-like object open for reading
   text or binary data, as appropriate for the specific codec.

   The "StreamReader" may implement different error handling schemes
   by providing the *errors* keyword argument. See Gestionnaires
   d'erreurs for the standard error handlers the underlying stream
   codec may support.

   The *errors* argument will be assigned to an attribute of the same
   name. Assigning to this attribute makes it possible to switch
   between different error handling strategies during the lifetime of
   the "StreamReader" object.

   The set of allowed values for the *errors* argument can be extended
   with "register_error()".

   read([size[, chars[, firstline]]])

      Decodes data from the stream and returns the resulting object.

      The *chars* argument indicates the number of decoded code points
      or bytes to return. The "read()" method will never return more
      data than requested, but it might return less, if there is not
      enough available.

      The *size* argument indicates the approximate maximum number of
      encoded bytes or code points to read for decoding. The decoder
      can modify this setting as appropriate. The default value -1
      indicates to read and decode as much as possible. This parameter
      is intended to prevent having to decode huge files in one step.

      The *firstline* flag indicates that it would be sufficient to
      only return the first line, if there are decoding errors on
      later lines.

      The method should use a greedy read strategy meaning that it
      should read as much data as is allowed within the definition of
      the encoding and the given size, e.g.  if optional encoding
      endings or state markers are available on the stream, these
      should be read too.

   readline([size[, keepends]])

      Read one line from the input stream and return the decoded data.

      *size*, if given, is passed as size argument to the stream's
      "read()" method.

      If *keepends* is false line-endings will be stripped from the
      lines returned.

   readlines([sizehint[, keepends]])

      Read all lines available on the input stream and return them as
      a list of lines.

      Line-endings are implemented using the codec's "decode()" method
      and are included in the list entries if *keepends* is true.

      *sizehint*, if given, is passed as the *size* argument to the
      stream's "read()" method.

   reset()

      Resets the codec buffers used for keeping state.

      Note that no stream repositioning should take place. This method
      is primarily intended to be able to recover from decoding
      errors.

In addition to the above methods, the "StreamReader" must also inherit
all other methods and attributes from the underlying stream.


StreamReaderWriter Objects
~~~~~~~~~~~~~~~~~~~~~~~~~~

The "StreamReaderWriter" is a convenience class that allows wrapping
streams which work in both read and write modes.

The design is such that one can use the factory functions returned by
the "lookup()" function to construct the instance.

class codecs.StreamReaderWriter(stream, Reader, Writer, errors='strict')

   Creates a "StreamReaderWriter" instance. *stream* must be a file-
   like object. *Reader* and *Writer* must be factory functions or
   classes providing the "StreamReader" and "StreamWriter" interface
   resp. Error handling is done in the same way as defined for the
   stream readers and writers.

"StreamReaderWriter" instances define the combined interfaces of
"StreamReader" and "StreamWriter" classes. They inherit all other
methods and attributes from the underlying stream.


StreamRecoder Objects
~~~~~~~~~~~~~~~~~~~~~

The "StreamRecoder" translates data from one encoding to another,
which is sometimes useful when dealing with different encoding
environments.

The design is such that one can use the factory functions returned by
the "lookup()" function to construct the instance.

class codecs.StreamRecoder(stream, encode, decode, Reader, Writer, errors='strict')

   Creates a "StreamRecoder" instance which implements a two-way
   conversion: *encode* and *decode* work on the frontend — the data
   visible to code calling "read()" and "write()", while *Reader* and
   *Writer* work on the backend — the data in *stream*.

   You can use these objects to do transparent transcodings, e.g.,
   from Latin-1 to UTF-8 and back.

   The *stream* argument must be a file-like object.

   The *encode* and *decode* arguments must adhere to the "Codec"
   interface. *Reader* and *Writer* must be factory functions or
   classes providing objects of the "StreamReader" and "StreamWriter"
   interface respectively.

   Error handling is done in the same way as defined for the stream
   readers and writers.

"StreamRecoder" instances define the combined interfaces of
"StreamReader" and "StreamWriter" classes. They inherit all other
methods and attributes from the underlying stream.


Encodings and Unicode
=====================

Strings are stored internally as sequences of code points in range
"0x0"--"0x10FFFF". (See **PEP 393** for more details about the
implementation.) Once a string object is used outside of CPU and
memory, endianness and how these arrays are stored as bytes become an
issue. As with other codecs, serialising a string into a sequence of
bytes is known as *encoding*, and recreating the string from the
sequence of bytes is known as *decoding*. There are a variety of
different text serialisation codecs, which are collectivity referred
to as *text encodings*.

The simplest text encoding (called "'latin-1'" or "'iso-8859-1'") maps
the code points 0--255 to the bytes "0x0"--"0xff", which means that a
string object that contains code points above "U+00FF" can't be
encoded with this codec. Doing so will raise a "UnicodeEncodeError"
that looks like the following (although the details of the error
message may differ): "UnicodeEncodeError: 'latin-1' codec can't encode
character '\u1234' in position 3: ordinal not in range(256)".

There's another group of encodings (the so called charmap encodings)
that choose a different subset of all Unicode code points and how
these code points are mapped to the bytes "0x0"--"0xff". To see how
this is done simply open e.g. "encodings/cp1252.py" (which is an
encoding that is used primarily on Windows). There's a string constant
with 256 characters that shows you which character is mapped to which
byte value.

All of these encodings can only encode 256 of the 1114112 code points
defined in Unicode. A simple and straightforward way that can store
each Unicode code point, is to store each code point as four
consecutive bytes. There are two possibilities: store the bytes in big
endian or in little endian order. These two encodings are called
"UTF-32-BE" and "UTF-32-LE" respectively. Their disadvantage is that
if e.g. you use "UTF-32-BE" on a little endian machine you will always
have to swap bytes on encoding and decoding. "UTF-32" avoids this
problem: bytes will always be in natural endianness. When these bytes
are read by a CPU with a different endianness, then bytes have to be
swapped though. To be able to detect the endianness of a "UTF-16" or
"UTF-32" byte sequence, there's the so called BOM ("Byte Order Mark").
This is the Unicode character "U+FEFF". This character can be
prepended to every "UTF-16" or "UTF-32" byte sequence. The byte
swapped version of this character ("0xFFFE") is an illegal character
that may not appear in a Unicode text. So when the first character in
an "UTF-16" or "UTF-32" byte sequence appears to be a "U+FFFE" the
bytes have to be swapped on decoding. Unfortunately the character
"U+FEFF" had a second purpose as a "ZERO WIDTH NO-BREAK SPACE": a
character that has no width and doesn't allow a word to be split. It
can e.g. be used to give hints to a ligature algorithm. With Unicode
4.0 using "U+FEFF" as a "ZERO WIDTH NO-BREAK SPACE" has been
deprecated (with "U+2060" ("WORD JOINER") assuming this role).
Nevertheless Unicode software still must be able to handle "U+FEFF" in
both roles: as a BOM it's a device to determine the storage layout of
the encoded bytes, and vanishes once the byte sequence has been
decoded into a string; as a "ZERO WIDTH NO-BREAK SPACE" it's a normal
character that will be decoded like any other.

There's another encoding that is able to encoding the full range of
Unicode characters: UTF-8. UTF-8 is an 8-bit encoding, which means
there are no issues with byte order in UTF-8. Each byte in a UTF-8
byte sequence consists of two parts: marker bits (the most significant
bits) and payload bits. The marker bits are a sequence of zero to four
"1" bits followed by a "0" bit. Unicode characters are encoded like
this (with x being payload bits, which when concatenated give the
Unicode character):

+-------------------------------------+------------------------------------------------+
| *Range*                             | Encoding                                       |
|=====================================|================================================|
| "U-00000000" ... "U-0000007F"       | 0xxxxxxx                                       |
+-------------------------------------+------------------------------------------------+
| "U-00000080" ... "U-000007FF"       | 110xxxxx 10xxxxxx                              |
+-------------------------------------+------------------------------------------------+
| "U-00000800" ... "U-0000FFFF"       | 1110xxxx 10xxxxxx 10xxxxxx                     |
+-------------------------------------+------------------------------------------------+
| "U-00010000" ... "U-0010FFFF"       | 11110xxx 10xxxxxx 10xxxxxx 10xxxxxx            |
+-------------------------------------+------------------------------------------------+

The least significant bit of the Unicode character is the rightmost x
bit.

As UTF-8 is an 8-bit encoding no BOM is required and any "U+FEFF"
character in the decoded string (even if it's the first character) is
treated as a "ZERO WIDTH NO-BREAK SPACE".

Without external information it's impossible to reliably determine
which encoding was used for encoding a string. Each charmap encoding
can decode any random byte sequence. However that's not possible with
UTF-8, as UTF-8 byte sequences have a structure that doesn't allow
arbitrary byte sequences. To increase the reliability with which a
UTF-8 encoding can be detected, Microsoft invented a variant of UTF-8
(that Python 2.5 calls ""utf-8-sig"") for its Notepad program: Before
any of the Unicode characters is written to the file, a UTF-8 encoded
BOM (which looks like this as a byte sequence: "0xef", "0xbb", "0xbf")
is written. As it's rather improbable that any charmap encoded file
starts with these byte values (which would e.g. map to

      LATIN SMALL LETTER I WITH DIAERESIS
      RIGHT-POINTING DOUBLE ANGLE QUOTATION MARK
      INVERTED QUESTION MARK

in iso-8859-1), this increases the probability that a "utf-8-sig"
encoding can be correctly guessed from the byte sequence. So here the
BOM is not used to be able to determine the byte order used for
generating the byte sequence, but as a signature that helps in
guessing the encoding. On encoding the utf-8-sig codec will write
"0xef", "0xbb", "0xbf" as the first three bytes to the file. On
decoding "utf-8-sig" will skip those three bytes if they appear as the
first three bytes in the file. In UTF-8, the use of the BOM is
discouraged and should generally be avoided.


Standard Encodings
==================

Python comes with a number of codecs built-in, either implemented as C
functions or with dictionaries as mapping tables. The following table
lists the codecs by name, together with a few common aliases, and the
languages for which the encoding is likely used. Neither the list of
aliases nor the list of languages is meant to be exhaustive. Notice
that spelling alternatives that only differ in case or use a hyphen
instead of an underscore are also valid aliases; therefore, e.g.
"'utf-8'" is a valid alias for the "'utf_8'" codec.

**CPython implementation detail:** Some common encodings can bypass
the codecs lookup machinery to improve performance. These optimization
opportunities are only recognized by CPython for a limited set of
(case insensitive) aliases: utf-8, utf8, latin-1, latin1, iso-8859-1,
iso8859-1, mbcs (Windows only), ascii, us-ascii, utf-16, utf16,
utf-32, utf32, and the same using underscores instead of dashes. Using
alternative aliases for these encodings may result in slower
execution.

Modifié dans la version 3.6: Optimization opportunity recognized for
us-ascii.

Many of the character sets support the same languages. They vary in
individual characters (e.g. whether the EURO SIGN is supported or
not), and in the assignment of characters to code positions. For the
European languages in particular, the following variants typically
exist:

* an ISO 8859 codeset

* a Microsoft Windows code page, which is typically derived from an
  8859 codeset, but replaces control characters with additional
  graphic characters

* an IBM EBCDIC code page

* an IBM PC code page, which is ASCII compatible

+-------------------+----------------------------------+----------------------------------+
| Codec             | Aliases                          | Languages                        |
|===================|==================================|==================================|
| *ascii*           | *646*, *us-ascii*                | Anglais                          |
+-------------------+----------------------------------+----------------------------------+
| *big5*            | *big5-tw*, *csbig5*              | Chinois Traditionnel             |
+-------------------+----------------------------------+----------------------------------+
| *big5hkscs*       | *big5-hkscs*, *hkscs*            | Chinois Traditionnel             |
+-------------------+----------------------------------+----------------------------------+
| *cp037*           | *IBM037*, *IBM039*               | Anglais                          |
+-------------------+----------------------------------+----------------------------------+
| *cp273*           | *273*, *IBM273*, *csIBM273*      | Allemand  Nouveau dans la        |
|                   |                                  | version 3.4.                     |
+-------------------+----------------------------------+----------------------------------+
| *cp424*           | *EBCDIC-CP-HE*, *IBM424*         | Hébreux                          |
+-------------------+----------------------------------+----------------------------------+
| *cp437*           | *437*, *IBM437*                  | Anglais                          |
+-------------------+----------------------------------+----------------------------------+
| *cp500*           | *EBCDIC-CP-BE*, *EBCDIC-CP-CH*,  | Europe de l'ouest                |
|                   | *IBM500*                         |                                  |
+-------------------+----------------------------------+----------------------------------+
| *cp720*           |                                  | Arabe                            |
+-------------------+----------------------------------+----------------------------------+
| *cp737*           |                                  | Grec                             |
+-------------------+----------------------------------+----------------------------------+
| *cp775*           | *IBM775*                         | Langues Baltiques                |
+-------------------+----------------------------------+----------------------------------+
| *cp850*           | *850*, *IBM850*                  | Europe de l'ouest                |
+-------------------+----------------------------------+----------------------------------+
| *cp852*           | *852*, *IBM852*                  | Europe centrale et Europe de     |
|                   |                                  | l'Est                            |
+-------------------+----------------------------------+----------------------------------+
| *cp855*           | *855*, *IBM855*                  | Bulgare, Biélorusse, Macédonien, |
|                   |                                  | Russe, Serbe                     |
+-------------------+----------------------------------+----------------------------------+
| *cp856*           |                                  | Hébreux                          |
+-------------------+----------------------------------+----------------------------------+
| *cp857*           | *857*, *IBM857*                  | Turc                             |
+-------------------+----------------------------------+----------------------------------+
| *cp858*           | *858*, *IBM858*                  | Europe de l'ouest                |
+-------------------+----------------------------------+----------------------------------+
| *cp860*           | *860*, *IBM860*                  | Portugais                        |
+-------------------+----------------------------------+----------------------------------+
| *cp861*           | *861*, *CP-IS*, *IBM861*         | Islandais                        |
+-------------------+----------------------------------+----------------------------------+
| *cp862*           | *862*, *IBM862*                  | Hébreux                          |
+-------------------+----------------------------------+----------------------------------+
| *cp863*           | *863*, *IBM863*                  | Canadien                         |
+-------------------+----------------------------------+----------------------------------+
| *cp864*           | *IBM864*                         | Arabe                            |
+-------------------+----------------------------------+----------------------------------+
| *cp865*           | *865*, *IBM865*                  | Danish, Norwegian                |
+-------------------+----------------------------------+----------------------------------+
| *cp866*           | *866*, *IBM866*                  | Russe                            |
+-------------------+----------------------------------+----------------------------------+
| *cp869*           | *869*, *CP-GR*, *IBM869*         | Grec                             |
+-------------------+----------------------------------+----------------------------------+
| *cp874*           |                                  | Thai                             |
+-------------------+----------------------------------+----------------------------------+
| *cp875*           |                                  | Grec                             |
+-------------------+----------------------------------+----------------------------------+
| *cp932*           | *932*, *ms932*, *mskanji*, *ms-  | Japanese                         |
|                   | kanji*                           |                                  |
+-------------------+----------------------------------+----------------------------------+
| *cp949*           | *949*, *ms949*, *uhc*            | Korean                           |
+-------------------+----------------------------------+----------------------------------+
| *cp950*           | *950*, *ms950*                   | Chinois Traditionnel             |
+-------------------+----------------------------------+----------------------------------+
| *cp1006*          |                                  | Urdu                             |
+-------------------+----------------------------------+----------------------------------+
| *cp1026*          | *ibm1026*                        | Turc                             |
+-------------------+----------------------------------+----------------------------------+
| *cp1125*          | *1125*, *ibm1125*, *cp866u*,     | Ukrainian  Nouveau dans la       |
|                   | *ruscii*                         | version 3.4.                     |
+-------------------+----------------------------------+----------------------------------+
| *cp1140*          | *ibm1140*                        | Europe de l'ouest                |
+-------------------+----------------------------------+----------------------------------+
| *cp1250*          | *windows-1250*                   | Europe centrale et Europe de     |
|                   |                                  | l'Est                            |
+-------------------+----------------------------------+----------------------------------+
| *cp1251*          | *windows-1251*                   | Bulgare, Biélorusse, Macédonien, |
|                   |                                  | Russe, Serbe                     |
+-------------------+----------------------------------+----------------------------------+
| *cp1252*          | *windows-1252*                   | Europe de l'ouest                |
+-------------------+----------------------------------+----------------------------------+
| *cp1253*          | *windows-1253*                   | Grec                             |
+-------------------+----------------------------------+----------------------------------+
| *cp1254*          | *windows-1254*                   | Turc                             |
+-------------------+----------------------------------+----------------------------------+
| *cp1255*          | *windows-1255*                   | Hébreux                          |
+-------------------+----------------------------------+----------------------------------+
| *cp1256*          | *windows-1256*                   | Arabe                            |
+-------------------+----------------------------------+----------------------------------+
| *cp1257*          | *windows-1257*                   | Langues Baltiques                |
+-------------------+----------------------------------+----------------------------------+
| *cp1258*          | *windows-1258*                   | Vietnamese                       |
+-------------------+----------------------------------+----------------------------------+
| *euc_jp*          | *eucjp*, *ujis*, *u-jis*         | Japanese                         |
+-------------------+----------------------------------+----------------------------------+
| *euc_jis_2004*    | *jisx0213*, *eucjis2004*         | Japanese                         |
+-------------------+----------------------------------+----------------------------------+
| *euc_jisx0213*    | *eucjisx0213*                    | Japanese                         |
+-------------------+----------------------------------+----------------------------------+
| *euc_kr*          | *euckr*, *korean*, *ksc5601*,    | Korean                           |
|                   | *ks_c-5601*, *ks_c-5601-1987*,   |                                  |
|                   | *ksx1001*, *ks_x-1001*           |                                  |
+-------------------+----------------------------------+----------------------------------+
| *gb2312*          | *chinese*, *csiso58gb231280*,    | Simplified Chinese               |
|                   | *euc-cn*, *euccn*,               |                                  |
|                   | *eucgb2312-cn*, *gb2312-1980*,   |                                  |
|                   | *gb2312-80*, *iso-ir-58*         |                                  |
+-------------------+----------------------------------+----------------------------------+
| *gbk*             | *936*, *cp936*, *ms936*          | Unified Chinese                  |
+-------------------+----------------------------------+----------------------------------+
| *gb18030*         | *gb18030-2000*                   | Unified Chinese                  |
+-------------------+----------------------------------+----------------------------------+
| *hz*              | *hzgb*, *hz-gb*, *hz-gb-2312*    | Simplified Chinese               |
+-------------------+----------------------------------+----------------------------------+
| *iso2022_jp*      | *csiso2022jp*, *iso2022jp*,      | Japanese                         |
|                   | *iso-2022-jp*                    |                                  |
+-------------------+----------------------------------+----------------------------------+
| *iso2022_jp_1*    | *iso2022jp-1*, *iso-2022-jp-1*   | Japanese                         |
+-------------------+----------------------------------+----------------------------------+
| *iso2022_jp_2*    | *iso2022jp-2*, *iso-2022-jp-2*   | Japanese, Korean, Simplified     |
|                   |                                  | Chinese, Western Europe, Greek   |
+-------------------+----------------------------------+----------------------------------+
| *iso2022_jp_2004* | *iso2022jp-2004*,                | Japanese                         |
|                   | *iso-2022-jp-2004*               |                                  |
+-------------------+----------------------------------+----------------------------------+
| *iso2022_jp_3*    | *iso2022jp-3*, *iso-2022-jp-3*   | Japanese                         |
+-------------------+----------------------------------+----------------------------------+
| *iso2022_jp_ext*  | *iso2022jp-ext*, *iso-2022-jp-   | Japanese                         |
|                   | ext*                             |                                  |
+-------------------+----------------------------------+----------------------------------+
| *iso2022_kr*      | *csiso2022kr*, *iso2022kr*,      | Korean                           |
|                   | *iso-2022-kr*                    |                                  |
+-------------------+----------------------------------+----------------------------------+
| *latin_1*         | *iso-8859-1*, *iso8859-1*,       | Europe de l'ouest                |
|                   | *8859*, *cp819*, *latin*,        |                                  |
|                   | *latin1*, *L1*                   |                                  |
+-------------------+----------------------------------+----------------------------------+
| *iso8859_2*       | *iso-8859-2*, *latin2*, *L2*     | Europe centrale et Europe de     |
|                   |                                  | l'Est                            |
+-------------------+----------------------------------+----------------------------------+
| *iso8859_3*       | *iso-8859-3*, *latin3*, *L3*     | Esperanto, Maltese               |
+-------------------+----------------------------------+----------------------------------+
| *iso8859_4*       | *iso-8859-4*, *latin4*, *L4*     | Langues Baltiques                |
+-------------------+----------------------------------+----------------------------------+
| *iso8859_5*       | *iso-8859-5*, *cyrillic*         | Bulgare, Biélorusse, Macédonien, |
|                   |                                  | Russe, Serbe                     |
+-------------------+----------------------------------+----------------------------------+
| *iso8859_6*       | *iso-8859-6*, *arabic*           | Arabe                            |
+-------------------+----------------------------------+----------------------------------+
| *iso8859_7*       | *iso-8859-7*, *greek*, *greek8*  | Grec                             |
+-------------------+----------------------------------+----------------------------------+
| *iso8859_8*       | *iso-8859-8*, *hebrew*           | Hébreux                          |
+-------------------+----------------------------------+----------------------------------+
| *iso8859_9*       | *iso-8859-9*, *latin5*, *L5*     | Turc                             |
+-------------------+----------------------------------+----------------------------------+
| *iso8859_10*      | *iso-8859-10*, *latin6*, *L6*    | Nordic languages                 |
+-------------------+----------------------------------+----------------------------------+
| *iso8859_11*      | *iso-8859-11*, *thai*            | Thai languages                   |
+-------------------+----------------------------------+----------------------------------+
| *iso8859_13*      | *iso-8859-13*, *latin7*, *L7*    | Langues Baltiques                |
+-------------------+----------------------------------+----------------------------------+
| *iso8859_14*      | *iso-8859-14*, *latin8*, *L8*    | Celtic languages                 |
+-------------------+----------------------------------+----------------------------------+
| *iso8859_15*      | *iso-8859-15*, *latin9*, *L9*    | Europe de l'ouest                |
+-------------------+----------------------------------+----------------------------------+
| *iso8859_16*      | *iso-8859-16*, *latin10*, *L10*  | South-Eastern Europe             |
+-------------------+----------------------------------+----------------------------------+
| *johab*           | *cp1361*, *ms1361*               | Korean                           |
+-------------------+----------------------------------+----------------------------------+
| *koi8_r*          |                                  | Russe                            |
+-------------------+----------------------------------+----------------------------------+
| *koi8_t*          |                                  | *Tajik*  Nouveau dans la version |
|                   |                                  | 3.5.                             |
+-------------------+----------------------------------+----------------------------------+
| *koi8_u*          |                                  | Ukrainian                        |
+-------------------+----------------------------------+----------------------------------+
| *kz1048*          | *kz_1048*, *strk1048_2002*,      | Kazakh  Nouveau dans la version  |
|                   | *rk1048*                         | 3.5.                             |
+-------------------+----------------------------------+----------------------------------+
| *mac_cyrillic*    | *maccyrillic*                    | Bulgare, Biélorusse, Macédonien, |
|                   |                                  | Russe, Serbe                     |
+-------------------+----------------------------------+----------------------------------+
| *mac_greek*       | *macgreek*                       | Grec                             |
+-------------------+----------------------------------+----------------------------------+
| *mac_iceland*     | *maciceland*                     | Islandais                        |
+-------------------+----------------------------------+----------------------------------+
| *mac_latin2*      | *maclatin2*, *maccentraleurope*  | Europe centrale et Europe de     |
|                   |                                  | l'Est                            |
+-------------------+----------------------------------+----------------------------------+
| *mac_roman*       | *macroman*, *macintosh*          | Europe de l'ouest                |
+-------------------+----------------------------------+----------------------------------+
| *mac_turkish*     | *macturkish*                     | Turc                             |
+-------------------+----------------------------------+----------------------------------+
| *ptcp154*         | *csptcp154*, *pt154*, *cp154*,   | Kazakh                           |
|                   | *cyrillic-asian*                 |                                  |
+-------------------+----------------------------------+----------------------------------+
| *shift_jis*       | *csshiftjis*, *shiftjis*,        | Japanese                         |
|                   | *sjis*, *s_jis*                  |                                  |
+-------------------+----------------------------------+----------------------------------+
| *shift_jis_2004*  | *shiftjis2004*, *sjis_2004*,     | Japanese                         |
|                   | *sjis2004*                       |                                  |
+-------------------+----------------------------------+----------------------------------+
| *shift_jisx0213*  | *shiftjisx0213*, *sjisx0213*,    | Japanese                         |
|                   | *s_jisx0213*                     |                                  |
+-------------------+----------------------------------+----------------------------------+
| *utf_32*          | *U32*, *utf32*                   | all languages                    |
+-------------------+----------------------------------+----------------------------------+
| *utf_32_be*       | *UTF-32BE*                       | all languages                    |
+-------------------+----------------------------------+----------------------------------+
| *utf_32_le*       | *UTF-32LE*                       | all languages                    |
+-------------------+----------------------------------+----------------------------------+
| *utf_16*          | *U16*, *utf16*                   | all languages                    |
+-------------------+----------------------------------+----------------------------------+
| *utf_16_be*       | *UTF-16BE*                       | all languages                    |
+-------------------+----------------------------------+----------------------------------+
| *utf_16_le*       | *UTF-16LE*                       | all languages                    |
+-------------------+----------------------------------+----------------------------------+
| *utf_7*           | *U7*, *unicode-1-1-utf-7*        | all languages                    |
+-------------------+----------------------------------+----------------------------------+
| *utf_8*           | U8, UTF, utf8, cp65001           | all languages                    |
+-------------------+----------------------------------+----------------------------------+
| *utf_8_sig*       |                                  | all languages                    |
+-------------------+----------------------------------+----------------------------------+

Modifié dans la version 3.4: The utf-16* and utf-32* encoders no
longer allow surrogate code points ("U+D800"--"U+DFFF") to be encoded.
The utf-32* decoders no longer decode byte sequences that correspond
to surrogate code points.

Modifié dans la version 3.8: "cp65001" is now an alias to "utf_8".


Python Specific Encodings
=========================

A number of predefined codecs are specific to Python, so their codec
names have no meaning outside Python. These are listed in the tables
below based on the expected input and output types (note that while
text encodings are the most common use case for codecs, the underlying
codec infrastructure supports arbitrary data transforms rather than
just text encodings). For asymmetric codecs, the stated meaning
describes the encoding direction.


Text Encodings
--------------

The following codecs provide "str" to "bytes" encoding and *bytes-like
object* to "str" decoding, similar to the Unicode text encodings.

+----------------------+-----------+-----------------------------+
| Codec                | Aliases   | Signification               |
|======================|===========|=============================|
| idna                 |           | Implement **RFC 3490**, see |
|                      |           | also "encodings.idna". Only |
|                      |           | "errors='strict'" is        |
|                      |           | supported.                  |
+----------------------+-----------+-----------------------------+
| mbcs                 | ansi,     | Windows only: Encode the    |
|                      | dbcs      | operand according to the    |
|                      |           | ANSI codepage (CP_ACP).     |
+----------------------+-----------+-----------------------------+
| oem                  |           | Windows only: Encode the    |
|                      |           | operand according to the    |
|                      |           | OEM codepage (CP_OEMCP).    |
|                      |           | Nouveau dans la version     |
|                      |           | 3.6.                        |
+----------------------+-----------+-----------------------------+
| palmos               |           | Encoding of PalmOS 3.5.     |
+----------------------+-----------+-----------------------------+
| punycode             |           | Implement **RFC 3492**.     |
|                      |           | Stateful codecs are not     |
|                      |           | supported.                  |
+----------------------+-----------+-----------------------------+
| raw_unicode_escape   |           | Latin-1 encoding with       |
|                      |           | "\uXXXX" and "\UXXXXXXXX"   |
|                      |           | for other code points.      |
|                      |           | Existing backslashes are    |
|                      |           | not escaped in any way. It  |
|                      |           | is used in the Python       |
|                      |           | pickle protocol.            |
+----------------------+-----------+-----------------------------+
| undefined            |           | Raise an exception for all  |
|                      |           | conversions, even empty     |
|                      |           | strings. The error handler  |
|                      |           | is ignored.                 |
+----------------------+-----------+-----------------------------+
| unicode_escape       |           | Encoding suitable as the    |
|                      |           | contents of a Unicode       |
|                      |           | literal in ASCII- encoded   |
|                      |           | Python source code, except  |
|                      |           | that quotes are not         |
|                      |           | escaped. Decode from        |
|                      |           | Latin-1 source code. Beware |
|                      |           | that Python source code     |
|                      |           | actually uses UTF-8 by      |
|                      |           | default.                    |
+----------------------+-----------+-----------------------------+

Modifié dans la version 3.8: "unicode_internal" codec is removed.


Binary Transforms
-----------------

The following codecs provide binary transforms: *bytes-like object* to
"bytes" mappings. They are not supported by "bytes.decode()" (which
only produces "str" output).

+------------------------+--------------------+--------------------------------+--------------------------------+
| Codec                  | Aliases            | Signification                  | Encoder / decoder              |
|========================|====================|================================|================================|
| base64_codec [1]       | base64, base_64    | Convert the operand to         | "base64.encodebytes()" /       |
|                        |                    | multiline MIME base64 (the     | "base64.decodebytes()"         |
|                        |                    | result always includes a       |                                |
|                        |                    | trailing "'\n'").  Modifié     |                                |
|                        |                    | dans la version 3.4: accepts   |                                |
|                        |                    | any *bytes-like object* as     |                                |
|                        |                    | input for encoding and         |                                |
|                        |                    | decoding                       |                                |
+------------------------+--------------------+--------------------------------+--------------------------------+
| bz2_codec              | bz2                | Compress the operand using     | "bz2.compress()" /             |
|                        |                    | bz2.                           | "bz2.decompress()"             |
+------------------------+--------------------+--------------------------------+--------------------------------+
| hex_codec              | hex                | Convert the operand to         | "binascii.b2a_hex()" /         |
|                        |                    | hexadecimal representation,    | "binascii.a2b_hex()"           |
|                        |                    | with two digits per byte.      |                                |
+------------------------+--------------------+--------------------------------+--------------------------------+
| quopri_codec           | quopri,            | Convert the operand to MIME    | "quopri.encode()" with         |
|                        | quotedprintable,   | quoted printable.              | "quotetabs=True" /             |
|                        | quoted_printable   |                                | "quopri.decode()"              |
+------------------------+--------------------+--------------------------------+--------------------------------+
| uu_codec               | uu                 | Convert the operand using      | "uu.encode()" / "uu.decode()"  |
|                        |                    | uuencode.                      |                                |
+------------------------+--------------------+--------------------------------+--------------------------------+
| zlib_codec             | zip, zlib          | Compress the operand using     | "zlib.compress()" /            |
|                        |                    | gzip.                          | "zlib.decompress()"            |
+------------------------+--------------------+--------------------------------+--------------------------------+

[1] In addition to *bytes-like objects*, "'base64_codec'" also accepts
    ASCII-only instances of "str" for decoding

Nouveau dans la version 3.2: Restoration of the binary transforms.

Modifié dans la version 3.4: Restoration of the aliases for the binary
transforms.


Text Transforms
---------------

The following codec provides a text transform: a "str" to "str"
mapping. It is not supported by "str.encode()" (which only produces
"bytes" output).

+----------------------+-----------+-----------------------------+
| Codec                | Aliases   | Signification               |
|======================|===========|=============================|
| rot_13               | rot13     | Return the Caesar-cypher    |
|                      |           | encryption of the operand.  |
+----------------------+-----------+-----------------------------+

Nouveau dans la version 3.2: Restoration of the "rot_13" text
transform.

Modifié dans la version 3.4: Restoration of the "rot13" alias.


"encodings.idna" --- Internationalized Domain Names in Applications
===================================================================

This module implements **RFC 3490** (Internationalized Domain Names in
Applications) and **RFC 3492** (Nameprep: A Stringprep Profile for
Internationalized Domain Names (IDN)). It builds upon the "punycode"
encoding and "stringprep".

If you need the IDNA 2008 standard from **RFC 5891** and **RFC 5895**,
use the third-party *idna module <https://pypi.org/project/idna/>_*.

These RFCs together define a protocol to support non-ASCII characters
in domain names. A domain name containing non-ASCII characters (such
as "www.Alliancefrançaise.nu") is converted into an ASCII-compatible
encoding (ACE, such as "www.xn--alliancefranaise-npb.nu"). The ACE
form of the domain name is then used in all places where arbitrary
characters are not allowed by the protocol, such as DNS queries, HTTP
*Host* fields, and so on. This conversion is carried out in the
application; if possible invisible to the user: The application should
transparently convert Unicode domain labels to IDNA on the wire, and
convert back ACE labels to Unicode before presenting them to the user.

Python supports this conversion in several ways:  the "idna" codec
performs conversion between Unicode and ACE, separating an input
string into labels based on the separator characters defined in
**section 3.1 of RFC 3490** and converting each label to ACE as
required, and conversely separating an input byte string into labels
based on the "." separator and converting any ACE labels found into
unicode. Furthermore, the "socket" module transparently converts
Unicode host names to ACE, so that applications need not be concerned
about converting host names themselves when they pass them to the
socket module. On top of that, modules that have host names as
function parameters, such as "http.client" and "ftplib", accept
Unicode host names ("http.client" then also transparently sends an
IDNA hostname in the *Host* field if it sends that field at all).

When receiving host names from the wire (such as in reverse name
lookup), no automatic conversion to Unicode is performed: applications
wishing to present such host names to the user should decode them to
Unicode.

The module "encodings.idna" also implements the nameprep procedure,
which performs certain normalizations on host names, to achieve case-
insensitivity of international domain names, and to unify similar
characters. The nameprep functions can be used directly if desired.

encodings.idna.nameprep(label)

   Return the nameprepped version of *label*. The implementation
   currently assumes query strings, so "AllowUnassigned" is true.

encodings.idna.ToASCII(label)

   Convert a label to ASCII, as specified in **RFC 3490**.
   "UseSTD3ASCIIRules" is assumed to be false.

encodings.idna.ToUnicode(label)

   Convert a label to Unicode, as specified in **RFC 3490**.


"encodings.mbcs" --- Windows ANSI codepage
==========================================

This module implements the ANSI codepage (CP_ACP).

Disponibilité : Windows uniquement.

Modifié dans la version 3.3: Support any error handler.

Modifié dans la version 3.2: Before 3.2, the *errors* argument was
ignored; "'replace'" was always used to encode, and "'ignore'" to
decode.


"encodings.utf_8_sig" --- UTF-8 codec with BOM signature
========================================================

This module implements a variant of the UTF-8 codec. On encoding, a
UTF-8 encoded BOM will be prepended to the UTF-8 encoded bytes. For
the stateful encoder this is only done once (on the first write to the
byte stream). On decoding, an optional UTF-8 encoded BOM at the start
of the data will be skipped.
