weibo表无法写入,user表正常
- 问:请您指明哪个版本出了bug(github版/PyPi版/全部)?
答:github
- 问:您使用的是否是最新的程序(是/否)?
答:是
- 问:爬取任意用户都会复现此bug吗(是/否)?
答:是
- 问:若只有爬特定微博时才出bug,能否提供出错微博的weibo_id或url(非必填)?
答:1987241375 蘑菇小牙牙 2026-02-24 14:15 或者 5371906414 佩奇线报 2026-02-24 14:15
- 问:若您已提供出错微博的weibo_id或url,可忽略此内容,否则能否提供出错账号的user_id及您配置的since_date,方便我们定位出错微博(非必填)?
答:
{ "user_id_list": "user_id_list.txt", "filter": 0, "since_date": "2026-02-24", "end_date": "now", "random_wait_pages": [ 1, 5 ], "random_wait_seconds": [ 6, 10 ], "global_wait": [ [ 1000, 3600 ], [ 500, 2000 ] ], "write_mode": [ "mysql" ], "pic_download": 0, "video_download": 0, "file_download_timeout": [ 5, 5, 10 ], "result_dir_name": 0, "cookie": "my cookie...", "mysql_config": { "host": "127.0.0.1", "port": 3306, "user": "root", "password": "root", "charset": "utf8mb4" }, "kafka_config": { "bootstrap-server": "127.0.0.1:9092", "weibo_topics": [ "spider_weibo" ], "user_topics": [ "spider_weibo" ] }, "sqlite_config": "weibo.db", "mongo_config": { "connection_string": "mongodb://admin:password@localhost:27017/weibo", "dba_name": "", "dba_password": "" }, "post_config": { "api_url": "", "api_token": "" } }
- 问:如果方便,请您描述bug详情,如果代码报错,最好附上错误提示。
答:
ubuntu @ ubuntu in ~/www/weiboSpider on git:master x [14:14:03] $ python3 -m weibo_spider 用户昵称: 蘑菇小牙牙 用户id: 1987241375 微博数: 539367 关注数: 2292 粉丝数: 970000
蘑菇小牙牙信息写入MySQL数据库完毕
Progress: 0%| | 0/53904 [00:00<?, ?it/s]/home/ubuntu/www/weiboSpider/weibo_spider/parser/page_parser.py:45: FutureWarning: Truth-testing of elements was a source of confusion and will always return True in future versions. Use specific 'len(elem)' or 'elem is not None' test instead. if self.selector: 东 反馈小鹿蓝蓝高钙高铁高锌DHA苏打脆片80g弹4立减,凑后【0.95】小鹿蓝蓝高钙高铁高锌DHA苏打脆片宝宝零食儿童苏打脆片/海苔味/80g凑单拍3件,同店介意慎网页链接 [组图共2张] 原图 微博发布位置:无 发布时间:2026-02-24 13:19 发布工具:蘑菇小牙牙的小店 点赞数:20 转发数:0 评论数:34 url:https://weibo.cn/comment/QtiLt3Yye
------------------------------已获取蘑菇小牙牙(1987241375)的第1页微博------------------------------ dict can not be used as parameter Traceback (most recent call last): File "/home/ubuntu/www/weiboSpider/weibo_spider/writer/mysql_writer.py", line 72, in _mysql_insert cursor.executemany( File "/home/ubuntu/.local/lib/python3.10/site-packages/pymysql/cursors.py", line 191, in executemany self.rowcount = sum(self.execute(query, arg) for arg in args) File "/home/ubuntu/.local/lib/python3.10/site-packages/pymysql/cursors.py", line 191, in self.rowcount = sum(self.execute(query, arg) for arg in args) File "/home/ubuntu/.local/lib/python3.10/site-packages/pymysql/cursors.py", line 151, in execute query = self.mogrify(query, args) File "/home/ubuntu/.local/lib/python3.10/site-packages/pymysql/cursors.py", line 129, in mogrify query = query % self._escape_args(args, conn) File "/home/ubuntu/.local/lib/python3.10/site-packages/pymysql/cursors.py", line 102, in _escape_args return tuple(conn.literal(arg) for arg in args) File "/home/ubuntu/.local/lib/python3.10/site-packages/pymysql/cursors.py", line 102, in return tuple(conn.literal(arg) for arg in args) File "/home/ubuntu/.local/lib/python3.10/site-packages/pymysql/connections.py", line 542, in literal return self.escape(obj, self.encoders) File "/home/ubuntu/.local/lib/python3.10/site-packages/pymysql/connections.py", line 535, in escape return converters.escape_item(obj, self.charset, mapping=mapping) File "/home/ubuntu/.local/lib/python3.10/site-packages/pymysql/converters.py", line 23, in escape_item val = encoder(val, charset, mapping) File "/home/ubuntu/.local/lib/python3.10/site-packages/pymysql/converters.py", line 30, in escape_dict raise TypeError("dict can not be used as parameter") TypeError: dict can not be used as parameter 1条微博写入MySQL数据库完毕 Progress: 0%| | 0/53904 [00:03<?, ?it/s] 共爬取1条微博 信息抓取完毕
ubuntu @ ubuntu in ~/www/weiboSpider on git:master x [14:29:22]
$ python3 --version
Python 3.10.12
试过mysql5和mysql8都一样是user表能写入,weibo表无法写入
Source: dataabc/weiboSpider