[{"data":1,"prerenderedAt":63},["ShallowReactive",2],{"article-33":3},{"code":4,"msg":5,"data":6,"count":16},200,"查询成功",{"id":7,"title":8,"keywords":9,"description":10,"category_id":11,"content":12,"body_html":13,"thumb_up":14,"clicks":15,"sort":16,"remark":17,"status":14,"is_open":14,"is_deleted":16,"is_top":16,"is_recommend":16,"create_time":18,"update_time":19,"image_id":20,"url":13,"member_id":16,"cate_name":21,"prev":22,"next":25,"tags":28,"words":37,"read_time":11,"comments":14,"cover":38,"relevant":39},33,"多线程爬去英雄联盟所有英雄的皮肤","python,多线程,爬虫","本文章利用Python中的多线程爬去英雄联盟官网所有英雄的皮肤。",5,"本文章利用Python中的多线程爬去英雄联盟官网所有英雄的皮肤。\n\n该案例需要的库有：\n\n```python\nrequests,re,json,threading\n```\n##### 一、目标网站\n\n```python\nhttps:\u002F\u002Flol.qq.com\u002Fdata\u002Finfo-heros.shtml\n```\n查找英雄皮肤的图片地址，并且分析格式\n\n```python\nhttps:\u002F\u002Fossweb-img.qq.com\u002Fimages\u002Flol\u002Fweb201310\u002Fskin\u002Fbig266000.jpg #剑魔默认皮肤地址\nhttps:\u002F\u002Fossweb-img.qq.com\u002Fimages\u002Flol\u002Fweb201310\u002Fskin\u002Fbig266001.jpg #剑魔仲裁圣骑皮肤地址\nhttps:\u002F\u002Fossweb-img.qq.com\u002Fimages\u002Flol\u002Fweb201310\u002Fskin\u002Fbig84000.jpg  #阿卡丽默认皮肤地址\nhttps:\u002F\u002Fossweb-img.qq.com\u002Fimages\u002Flol\u002Fweb201310\u002Fskin\u002Fbig84005.jpg  #阿卡丽腥红之月鬼武姬皮肤地址\n```\n仔细分析发现皮肤连接中只有big后面的数据不一致，其他都一样。则说明所有皮肤的地址是\n\n```python\nhttps:\u002F\u002Fossweb-img.qq.com\u002Fimages\u002Flol\u002Fweb201310\u002Fskin\u002Fbig+数字+.jpg\n```\n分析266000和266001，84000和84005数字，每个英雄前几位数字相同，后三位都是从000开始的，分析得到后三位数字是每个英雄皮肤的序号\n\n前面的数字是英雄id，只要找到所有英雄的id我们就可以找到所有英雄的所有皮肤地址。\n\n##### 二、查找所有英雄的id\n\n1、分析网站前端源码查找\n```python\n没有找到\n```\n2、分析网站所有后端请求地址\n```python\n也没有找到\n```\n3、分析网站的静态资源JavaScript\n```python \n终于发现一个js文件存放英雄id\n```\n```python\nhttps:\u002F\u002Flol.qq.com\u002Fbiz\u002Fhero\u002Fchampion.js\n```\n英雄id存放地址\n\n\n![](https:\u002F\u002Ftp.myong.top\u002Fstorage\u002Fckeditor\u002F20190128\u002F0533feb0519692bc0bc2dee49be64509.jpg)\n##### 三、程序编写\n\n现在皮肤地址有了，英雄的id也有了，开始程序代码编写\n\n1、引入所需要的库\n\n```python\nimport requests\nimport re,json,threading\n```\n2、程序入口\n\n```python\ndef main():\n    #所用英雄id请求地址\n    url = 'https:\u002F\u002Flol.qq.com\u002Fbiz\u002Fhero\u002Fchampion.js'\n    ids_dict = get_lol_ids_dict(url)\n    get_img(ids_dict)\n    \nif __name__ == \"__main__\":\n    main()\n```\n3、解析js文件提取英雄id和名称\n\n利用正则提取id和name\n\n```python\ndef get_lol_ids_dict(url):\n    headers = {\n        'User-Agent':'Mozilla\u002F5.0 (Windows NT 6.1; WOW64; rv:64.0) Gecko\u002F20100101 Firefox\u002F64.0'\n    }\n    response = requests.get(url=url, headers=headers).text\n    #\n    ids_json = re.search('\"keys\":(.*),\"data\"', response).group(1)#group 第一个小括号内容\n    ids_dict = json.loads(ids_json)\n    return ids_dict\n```\n4、拼接所有皮肤连接地址并保存\n\n```python\ndef get_img(ids_dict):\n    start_url = 'https:\u002F\u002Fossweb-img.qq.com\u002Fimages\u002Flol\u002Fweb201310\u002Fskin\u002Fbig'\n    for id,name in ids_dict.items():\n        #print(id,name)\n        for k in range(13):#最多12张皮肤\n            url = start_url + id + '%03d'%k + '.jpg'\n            #print(url)\n            response = requests.get(url = url)\n            if response.status_code == 200:\n                with open('img\u002F%s%d.jpg'%(name, k), 'wb') as f:\n                    f.write(response.content) \n```\n该方法下载皮肤速度太慢，改进程序，利用多线程下载皮肤\n\n```python\ndef get_img(ids_dict):\n    start_url = 'https:\u002F\u002Fossweb-img.qq.com\u002Fimages\u002Flol\u002Fweb201310\u002Fskin\u002Fbig'\n    \n    index   = 0\n    for id,name in ids_dict.items():\n        #print(id,name)\n        threads = []#多线程\n        for k in range(13):#最多12张皮肤\n            url = start_url + id + '%03d'%k + '.jpg'\n            #print(url)\n            response = requests.get(url = url)\n            if response.status_code == 200:\n                threads.append(threading.Thread(target = save_img, args = (url, name, k, index)))\n                index += 1\n                #td = threading.Thread(target=save_img, args=(url,name, k))\n                #td.start()\n        for td in threads:\n            td.setDaemon(True)\n            td.start()\n        td.join() #父线程，等待所有线程结束\n```\n5、保存皮肤方法\n\n```python\ndef save_img(url, name, k, index):\n    response = requests.get(url = url)\n    print(url)\n    print('正在下载第%d张是【%s】的第%d张皮肤'%(index, name, k))\n    with open('img\u002F%s%d.jpg'%(name, k), 'wb') as f:\n        f.write(response.content) \n```\n到这一步我们的爬去皮肤的程序就结束了。\n",null,1,530,0,"","2019-01-28 11:58:42","2026-04-19 15:15:50",124,"Python",{"id":23,"title":24},32,"Python爬虫的基本思路",{"id":26,"title":27},34,"Python基础整理之列表常见操作",[29,31,34],{"id":30,"name":21},27,{"id":32,"name":33},51,"爬虫",{"id":35,"name":36},52,"多线程",2082,"https:\u002F\u002Ftp.myong.top\u002Fstorage\u002Farticle\u002F69\u002F38924b4eb5e960ecdb0961eb143096.jpg",[40,45,50,55,58],{"id":41,"title":42,"create_time":43,"description":44},95,"Django启用memcache缓存","2020-10-23 11:17:27","在动态网站中,用户所有的请求,服务器都会去数据库中进行相应的增,删,查,改,渲染模板,执行业务逻辑,最后生成用户看到的页面.当一个网站的用户访问量很大的时候,每一次的的后台操作,都会消耗很多的服务端资源,所以必须使用缓存来减轻后端服务器的压力.",{"id":46,"title":47,"create_time":48,"description":49},91,"用Python实现编程语言 20 年的动态排行榜","2020-06-24 16:06:54","爬取一下，自2001年5月至今，TIOBE 编程语言排行榜上编程语言的变化情况，看一下在接近20年的时间里，编程语言的热度是如何变化的。\r\n",{"id":51,"title":52,"create_time":53,"description":54},100,"re正则表达式","2021-07-10 18:09:52","正则表达式是一个特殊的字符序列，它能帮助你方便的检查一个字符串是否与某种模式匹配，本文罗列re正则表达式中，常用的匹配符号。",{"id":26,"title":27,"create_time":56,"description":57},"2019-02-10 16:53:55","列表是最常用的Python数据类型，它可以作为一个方括号内的逗号分隔值出现。列表的数据项不需要具有相同的类型",{"id":59,"title":60,"create_time":61,"description":62},63,"Python与mysqldump的数据库备份","2019-10-14 15:35:33","mysqldump是备份MySQL数据库的一种好工具。它相对于用phpmyadmin等备份工具更加快速，又避免受php.ini等的限制，在windows系统下还可以结合计划任务实现定时远程备份数据库",1783431668429]