#210·MegaParse

Charmap codec issue

Author: ghostCreated Dec 24, 2024Updated Jan 13, 2025

I was face this issue,


Traceback (most recent call last):
  File "D:\Devolvedai_work\Devolved_AI_Data_Preparation\perser_experiments\test.py", line 9, in <module>
    megaparse.save(r"D:/Devolvedai_work/Devolved_AI_Data_Preparation/perser_experiments/1.md")
  File "D:\Devolvedai_work\Devolved_AI_Data_Preparation\venv\Lib\site-packages\megaparse\megaparse.py", line 133, in save
    f.write(self.last_parsed_document)
  File "C:\Users\hp\AppData\Local\Programs\Python\Python311\Lib\encodings\cp1252.py", line 19, in encode
    return codecs.charmap_encode(input,self.errors,encoding_table)[0]
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
UnicodeEncodeError: 'charmap' codec can't encode character '\u0338' in position 12572: character maps to <undefined>

It solved by creating a save function manually. Here i just added the encoding parameters


def save(file_path:str) -> None:
    os.makedirs(os.path.dirname(file_path), exist_ok=True)
    with open(file_path, "w+", encoding="utf-8") as f:  # Specify UTF-8 encoding
        f.write(megaparse.last_parsed_document)