Module parserino.encoding

Encodings other than UTF-8.

Document takes UTF-8. For html in another encoding (old pages, files saved as windows-1252, UTF-16, ...) convert the bytes first:

import parserino, parserino.encoding;
auto doc = Document(toUtf8(cast(const(ubyte)[]) std.file.read("page.html")));

toUtf8 finds the encoding as the HTML standard does ("encoding sniffing"): the byte order mark, then the <meta charset> (or http-equiv=content-type) in the first 1024 bytes. If nothing says it, valid UTF-8 is UTF-8, anything else windows-1252 (what browsers use for western pages). With the Content-Type of an HTTP response, pass its charset: it wins over the <meta>.

Supported

UTF-8, UTF-16LE, UTF-16BE, windows-1252 (also for the labels iso-8859-1, latin1, us-ascii, as the Encoding standard says), ISO-8859-2, windows-1250, windows-1251.

Functions

Name	Description
`encodingForLabel(label)`	The canonical name of an encoding label ("latin1" is "windows-1252"), or null if not supported
`sniffEncoding(bytes, transportCharset)`	The encoding of the html in `bytes`, as its canonical name ("UTF-8", "windows-1252", ...). `transportCharset` is the charset of the HTTP `Content-Type`, if known: it comes after the BOM and before the `<meta>`.
`toUtf8(bytes, encoding, transportCharset)`	Convert html to UTF-8. `encoding` is a label ("utf-8", "latin1", "windows-1251", ...); if null it's found with `sniffEncoding` (`transportCharset` is the charset of the HTTP Content-Type, if known). Invalid sequences become U+FFFD. It throws `ParserinoException` for unsupported encodings.