TWINT missingimport related to cchardet, cannot build wheel
Issue Template
Please use this template!
Initial Check
If the issue is a request please specify that it is a request in the title (Example: [REQUEST] more features). If this is a question regarding 'twint' please specify that it's a question in the title (Example: [QUESTION] What is x?). Please only submit issues related to 'twint'. Thanks.
Make sure you've checked the following:
- [] Python version is 3.6;
- [] Updated Twint with
pip3 install --user --upgrade -e git+https://github.com/twintproject/twint.git@origin/master#egg=twint; - [] I have searched the issues and there are no duplicates of this issue/question/request.
Command Ran
py -m pip install cchardet AND py -m pip install twing
Description of Issue
cchardet has not been updated for a long time so it appears to be working for python 2, cannot build wheel
Environment Details
Windows, inside command prompt
Command prompt error details are as follows...
C:\Users>py -m pip install --user --upgrade -e git+https://github.com/twintproject/twint.git@origin/master#egg=twint Obtaining twint from git+https://github.com/twintproject/twint.git@origin/master#egg=twint Cloning https://github.com/twintproject/twint.git (to revision origin/master) to c:\\users\src\twint Running command git clone --filter=blob:none --quiet https://github.com/twintproject/twint.git 'C:\Users\src\twint' WARNING: Did not find branch or tag 'origin/master', assuming revision or ref. Running command git checkout -q origin/master Resolved https://github.com/twintproject/twint.git to commit origin/master Preparing metadata (setup.py) ... done Collecting aiohttp Using cached aiohttp-3.8.1-cp310-cp310-win_amd64.whl (555 kB) Collecting aiodns Using cached aiodns-3.0.0-py3-none-any.whl (5.0 kB) Requirement already satisfied: beautifulsoup4 in c:\\users\appdata\local\programs\python\python310\lib\site-packages (from twint) (4.10.0) Collecting cchardet Using cached cchardet-2.1.7.tar.gz (653 kB) Preparing metadata (setup.py) ... done Requirement already satisfied: dataclasses in c:\\users\appdata\local\programs\python\python310\lib\site-packages (from twint) (0.6) Collecting elasticsearch Using cached elasticsearch-8.1.3-py3-none-any.whl (373 kB) Requirement already satisfied: pysocks in c:\\users\appdata\local\programs\python\python310\lib\site-packages (from twint) (1.7.1) Requirement already satisfied: pandas in c:\\users\appdata\local\programs\python\python310\lib\site-packages (from twint) (1.3.5) Collecting aiohttp_socks Using cached aiohttp_socks-0.7.1-py3-none-any.whl (9.3 kB) Collecting schedule Using cached schedule-1.1.0-py2.py3-none-any.whl (10 kB) Collecting geopy Using cached geopy-2.2.0-py3-none-any.whl (118 kB) Requirement already satisfied: fake-useragent in c:\\users\appdata\local\programs\python\python310\lib\site-packages (from twint) (0.1.11) Collecting googletransx Using cached googletransx-2.4.2-py3-none-any.whl Collecting pycares>=4.0.0 Using cached pycares-4.1.2-cp310-cp310-win_amd64.whl (83 kB) Requirement already satisfied: attrs>=17.3.0 in c:\\users\appdata\local\programs\python\python310\lib\site-packages (from aiohttp->twint) (21.4.0) Collecting multidict<7.0,>=4.5 Using cached multidict-6.0.2-cp310-cp310-win_amd64.whl (27 kB) Collecting frozenlist>=1.1.1 Using cached frozenlist-1.3.0-cp310-cp310-win_amd64.whl (33 kB) Collecting async-timeout<5.0,>=4.0.0a3 Using cached async_timeout-4.0.2-py3-none-any.whl (5.8 kB) Collecting yarl<2.0,>=1.0 Using cached yarl-1.7.2-cp310-cp310-win_amd64.whl (122 kB) Requirement already satisfied: charset-normalizer<3.0,>=2.0 in c:\\users\appdata\local\programs\python\python310\lib\site-packages (from aiohttp->twint) (2.0.12) Collecting aiosignal>=1.1.2 Using cached aiosignal-1.2.0-py3-none-any.whl (8.2 kB) Requirement already satisfied: python-socks[asyncio]<3.0.0,>=2.0.0 in c:\\users\appdata\local\programs\python\python310\lib\site-packages (from aiohttp_socks->twint) (2.0.3) Requirement already satisfied: soupsieve>1.2 in c:\\users\appdata\local\programs\python\python310\lib\site-packages (from beautifulsoup4->twint) (2.3.1) Collecting elastic-transport<9,>=8 Using cached elastic_transport-8.1.2-py3-none-any.whl (59 kB) Requirement already satisfied: geographiclib<2,>=1.49 in c:\\users\appdata\local\programs\python\python310\lib\site-packages (from geopy->twint) (1.52) Requirement already satisfied: requests in c:\\users\appdata\local\programs\python\python310\lib\site-packages (from googletransx->twint) (2.27.1) Requirement already satisfied: python-dateutil>=2.7.3 in c:\\users\appdata\roaming\python\python310\site-packages (from pandas->twint) (2.8.2) Requirement already satisfied: numpy>=1.21.0 in c:\\users\appdata\local\programs\python\python310\lib\site-packages (from pandas->twint) (1.21.5) Requirement already satisfied: pytz>=2017.3 in c:\\users\appdata\local\programs\python\python310\lib\site-packages (from pandas->twint) (2021.3) Requirement already satisfied: urllib3<2,>=1.26.2 in c:\\users\appdata\local\programs\python\python310\lib\site-packages (from elastic-transport<9,>=8->elasticsearch->twint) (1.26.8) Requirement already satisfied: certifi in c:\\users\appdata\local\programs\python\python310\lib\site-packages (from elastic-transport<9,>=8->elasticsearch->twint) (2021.10.8) Requirement already satisfied: cffi>=1.5.0 in c:\\users\appdata\local\programs\python\python310\lib\site-packages (from pycares>=4.0.0->aiodns->twint) (1.15.0) Requirement already satisfied: six>=1.5 in c:\\users\appdata\roaming\python\python310\site-packages (from python-dateutil>=2.7.3->pandas->twint) (1.16.0) Requirement already satisfied: idna>=2.0 in c:\\users\appdata\local\programs\python\python310\lib\site-packages (from yarl<2.0,>=1.0->aiohttp->twint) (3.3) Requirement already satisfied: pycparser in c:\\users\appdata\local\programs\python\python310\lib\site-packages (from cffi>=1.5.0->pycares>=4.0.0->aiodns->twint) (2.21) Building wheels for collected packages: cchardet Building wheel for cchardet (setup.py) ... error error: subprocess-exited-with-error
× python setup.py bdist_wheel did not run successfully. │ exit code: 1 ╰─> [12 lines of output] cythonize: ['src/cchardet\cchardet.pyx'] running bdist_wheel running build running build_py creating build creating build\lib.win-amd64-3.10 creating build\lib.win-amd64-3.10\cchardet copying src\cchardet\version.py -> build\lib.win-amd64-3.10\cchardet copying src\cchardet_init.py -> build\lib.win-amd64-3.10\cchardet running build_ext building 'cchardet._cchardet' extension error: Microsoft Visual C++ 14.0 or greater is required. Get it with "Microsoft C++ Build Tools": https://visualstudio.microsoft.com/visual-cpp-build-tools/ [end of output]
note: This error originates from a subprocess, and is likely not a problem with pip. ERROR: Failed building wheel for cchardet Running setup.py clean for cchardet Failed to build cchardet Installing collected packages: cchardet, schedule, multidict, geopy, frozenlist, elastic-transport, async-timeout, yarl, pycares, googletransx, elasticsearch, aiosignal, aiohttp, aiodns, aiohttp_socks, twint Running setup.py install for cchardet ... error error: subprocess-exited-with-error
× Running setup.py install for cchardet did not run successfully. │ exit code: 1 ╰─> [14 lines of output] cythonize: ['src/cchardet\cchardet.pyx'] running install C:\Users\Python\Python310\lib\site-packages\setuptools\command\install.py:34: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools. warnings.warn( running build running build_py creating build creating build\lib.win-amd64-3.10 creating build\lib.win-amd64-3.10\cchardet copying src\cchardet\version.py -> build\lib.win-amd64-3.10\cchardet copying src\cchardet_init.py -> build\lib.win-amd64-3.10\cchardet running build_ext building 'cchardet._cchardet' extension error: Microsoft Visual C++ 14.0 or greater is required. Get it with "Microsoft C++ Build Tools": https://visualstudio.microsoft.com/visual-cpp-build-tools/ [end of output]
note: This error originates from a subprocess, and is likely not a problem with pip. error: legacy-install-failure
× Encountered error while trying to install package. ╰─> cchardet
note: This is an issue with the package mentioned above, not pip. hint: See above for output from the failure.
Source: twintproject/twint