python-ftfy

If it's a mix of UTF-8 and a single-byte encoding, you can guess, because UTF-8 is nice enough to be obvious when you guessed wrong. The command-line version of ftfy [1] can handle a distinct encoding for each line with a near-zero rate of false positives. (No guarantees the output will be correct, especially if you read it as the wrong single-byte encoding initially, but it will be no worse than what you put in, because it won't replace correct text with incorrect text.) There is no JS version of ftfy, and there won't be until JS can represent all Unicode characters without surrogates. [1] https://github.com/LuminosoInsight/python-ftfy

ftfy (fixes text for you): https://github.com/LuminosoInsight/python-ftfy Auto-detects Unicode mistakes (particularly mojibake), and if there's enough information left to fix them, it fixes them. Particularly useful for Web scraping and dealing with Unicode that was incorrectly exported from Excel (which is nearly all Unicode exported from Excel).

The fortunate thing is, almost all of the broken sequences are unambiguous enough to be signs that the text should be encoded and then re-decoded as UTF-8. (This is not the case with any arbitrary encoding mixup -- if you mix up Big5 with EUC-JP, you might as well throw out your text and start over -- but it works for UTF-8 and the most common other encodings because UTF-8 is well-designed.) So if you want a Python library that can do this automatically with an extremely low rate of false positives: https://github.com/LuminosoInsight/python-ftfy