python爬虫,发送请求需要携带cookies

news/2024/7/19 9:30:13 标签: python, 爬虫, 开发语言

此网站,需要先注册登录!!!

页面源代码:


<!DOCTYPE html><html lang="zh-Hans"><head><meta charset="UTF-8"><meta http-equiv="x-ua-compatible" content="IE=edge"><meta name="viewport" content="width=device-width, initial-scale=1"><meta name="renderer" content="webkit"><meta name="keywords" content="股票,炒股,股票资讯,行情,财经,证券,投资,金融,港股,美股,基金,债券,期货,外汇,科创板,保险,雪球财经,雪球,雪球股票,投资社区,雪球投资"><meta name="description" content="雪球,聪明的投资者都在这里 - 4300万投资者都在用的投资社区和财富管理平台,沪深、港股、美股全球市场实时行情,公募私募股票基金债券免费热点资讯,与投资高手实战交流。支持股票基金在线开户,炒股、投资理财低佣金,交易安全、方便、快捷。提供选基工具、基金估值工具、基金定投、基金排名、指数估值、投资组合供投资者使用参考。"><meta name="format-detection" content="telephone=no"><meta name="baidu-site-verification" content="codeva-Q0pduO3YRC"><meta content="always" name="referrer"><link href="//assets.imedao.com/ugc/js/vue-common-6118e8a4ff.js" rel="preload" as="script"><link href="//assets.imedao.com/ugc/js/vue-home-1033b4aeaf.js" rel="preload" as="script"><link href="//xqdoc.imedao.com/17aebcfb84a145d33fc18679.ico" rel="shortcut icon"><link rel="stylesheet" href="//assets.imedao.com/ugc/css/component/index-6a2a1e3e87.css"><title>我的首页&nbsp;-&nbsp;雪球</title><link rel="stylesheet" href="//assets.imedao.com/ugc/css/home-99631a21fd.css"><link rel="stylesheet" href="//assets.imedao.com/ugc/css/im-ca8ee8fac5.css"><script>window.SNOWMAN_USER = {"id":6608945068,"name":null,"province":null,"city":null,"location":null,"description":null,"url":null,"domain":null,"gender":"n","verified":false,"type":"1","step":"null","profile":"/6608945068","recommend":null,"intro":null,"status":1,"following":false,"blocking":false,"subscribeable":false,"remark":null,"constrained":0,"screen_name":"用户6608945068","created_at":1705136454711,"followers_count":0,"friends_count":0,"status_count":0,"last_status_id":0,"blog_description":null,"st_color":"1","stocks_count":0,"cube_count":0,"donate_count":0,"verified_type":0,"verified_description":null,"verified_realname":false,"stock_status_count":null,"follow_me":false,"allow_all_stock":false,"name_pinyin":null,"screenname_pinyin":null,"group_ids":null,"common_count":0,"recommend_reason":null,"verified_infos":[],"select_company_background":null,"select_company_banners":null,"privacy_agreement":null,"ip_location":"河南","reg_time":1705136454722,"photo_domain":"//xavatar.imedao.com/","profile_image_url":"community/default/avatar.png,community/default/avatar.png!180x180.png,community/default/avatar.png!50x50.png,community/default/avatar.png!30x30.png"};</script><script>window.SNOWMAN_LOGIN = 1;
//- fix: 登录但是uid 为空问题
if(window.SNOWMAN_USER && !window.SNOWMAN_USER.id){
  window.location.reload();
}</script><script>window.SNB = {
  staticDomain: 'https://assets.imedao.com/'
};</script></head><body><div id="app"><div class="nav__placeholder"></div><nav class="nav stickyFixed"><div class="container signed_container"><div class="nav__lf"><a class="nav__logo" href="/" data-analytics="31" data-analytics-data="{type: '雪球图标'}"></a><div class="nav__menu"><a class="nav__menu__item active" href="/" data-analytics="31" data-analytics-data="{type: '首页'}">首页</a><nav-essence></nav-essence><!--a.nav__menu__item(href="/ask/square", data-analytics='31', data-analytics-data="{type: '问答'}") 问答--><div class="nav__menu__item">行情<i></i><div class="nav__dropdown__container"><i></i><div class="nav__dropdown"><a href="/hq">行情中心</a><a href="/stock/screener">筛选器</a><a href="/hq/cnew?market=CN&amp;first_name=0&amp;second_name=4">新股上市</a><a href="/p/discover">买什么</a></div></div></div><div class="nav__menu__item">交易<i></i><div class="nav__dropdown__container"><i></i><div class="nav__dropdown"><a href="https://broker.xueqiu.com" target="_blank">A股交易</a><a href="https://danjuanfunds.com" target="_blank">基金交易</a><a href="https://danjuanfunds.com/rn/pf-shelves/home?login_type=1" target="_blank">私募中心</a></div></div></div></div><search></search></div><div class="nav__rt"><div class="nav__user-info__main"><div class="nav__user-info__username"><div class="nav__user-info__base"><a href="javascript:;"><i class="iconfont">&#xe6f6;</i><span class="user-name">用户6608945068</span><nav-msg></nav-msg><cs-button :isLogin="1"></cs-button></a></div></div><nav-info></nav-info></div><nav-publish></nav-publish></div></div></nav><div class="home__main"><div class="user__container"><div class="user__col--lf"><a class="avatar" href="/u/6608945068" target="_blank" data-tooltip="false"><img src="//xavatar.imedao.com/community/default/avatar.png!180x180.png"></a><div class="user__col--name">用户6608945068</div><table class="user__col--friends"><td><a href="/center/#/friends"><strong>0</strong><br><span>关注</span></a></td><td><a href="/center/#/fans"><strong>0</strong><br><span>粉丝</span></a></td></table><ul class="user__control__pannel"><li><i class="iconfont">&#xe659;</i><a href="/">首页</a></li><li><i class="iconfont">&#xe65a;</i><a href="/center/#/status">讨论</a></li><li class="performance"><i class="iconfont">&#xe651;</i><a href="/performance">持仓盈亏</a></li><li><i class="iconfont">&#xe650;</i><a href="/center/#/fav">收藏</a></li></ul><ul class="user__control__pannel"><li><i class="iconfont">&#xe653;</i><a href="/center/#/likes">赞我的</a></li><li><i class="iconfont">&#xe655;</i><a href="/center/#/comments">讨论我的</a></li><li><i class="iconfont">&#xe657;</i><a href="/center/#/mention">提到我的</a></li><li><i class="iconfont">&#xe654;</i><a href="/center/#/quiz">我收到的问题</a></li></ul><ul class="user__control__pannel"><li><i class="iconfont">&#xe669;</i><a href="/6608945068/column">我的专栏</a></li><li><i class="iconfont">&#xe64f;</i><a href="/center/#/wallet">我的钱包</a></li></ul><ul class="user__control__pannel"><li><i class="iconfont">&#xe60d;</i><a href="/center/#/report">投诉中心</a></li></ul><ul class="xueqiu__menu"><li><i class="iconfont">&#xe652;</i><a href="/about" target="_blank">关于雪球</a></li><li><i class="iconfont">&#xe658;</i><a href="/dh" target="_blank">财经导航</a></li><li><i class="iconfont">&#xe648;</i><a href="/invest-edu" target="_blank">投资者教育</a></li><li><img class="iconfont_img" src="https://xqimg.imedao.com/1887627038f25f273fe5dafc.png"><a href="/about/faq#5_0" target="_blank">风险提示</a></li></ul></div><div class="user__col--middle"><div class="home__stock-index"></div><div class="snb_skeleton" id="skeleton__home__stock-index"><div class="home__stock-index__box"><div class="home__stock-index__item"><p>上证指数</p><div class="info"><div class="current">————</div><div class="growth">————</div></div><a class="chart"></a></div><div class="home__stock-index__item"><p>深证成指</p><div class="info"><div class="current">————</div><div class="growth">————</div></div><a class="chart"></a></div><div class="home__stock-index__item"><p>创业板指</p><div class="info"><div class="current">————</div><div class="growth">————</div></div><a class="chart"></a></div></div></div><index-svg></index-svg><div class="optional optional"></div><div class="snb_skeleton" id="skeleton__optional"><div class="optional__tabs__controls"><a class="tabs__control">自选股票</a><a class="tabs__control">自选基金</a><a class="tabs__control">自选组合</a></div><div class="optional__tabs__contents"><table class="optional__tb optional_stocks"><tr><th class="optional__category"></th></tr><tr> <td></td></tr><tr> <td></td></tr><tr> <td></td></tr><tr> <td></td></tr><tr> <td></td></tr><tr> <td></td></tr></table></div></div><optional></optional><div class="editor-container"><tiny-editor :config="{storage : true}"></tiny-editor></div><div class="snb_skeleton" id="skeleton__timeline"><div class="home-timeline"><div class="home-timeline-tabs"><a>全部</a><a>讨论</a><a>交易</a><a>新闻</a><a>公告</a></div><div class="status-list"><div class="timeline__item"><a class="avatar avatar-md"></a><div class="timeline__item__main"><div class="timeline__item__info"><div><a class="user-name">--------</a></div><a class="date-and-source">1分钟前 · 来自雪球</a></div><div class="timeline__item__bd"><div class="content content--description">------------------------------------------------------------------------------------------------------------------------------------------------------</div></div><div class="timeline__item__ft"><div class="timeline__item__control"></div></div></div></div><div class="timeline__item"><a class="avatar avatar-md"></a><div class="timeline__item__main"><div class="timeline__item__info"><div><a class="user-name">--------</a></div><a class="date-and-source">2分钟前 · 来自雪球</a></div><div class="timeline__item__bd"><div class="content content--description">------------------------------------------------------------------------------------------------------------------------------------------------------</div></div><div class="timeline__item__ft"><div class="timeline__item__control"></div></div></div></div><div class="timeline__item"><a class="avatar avatar-md"></a><div class="timeline__item__main"><div class="timeline__item__info"><div><a class="user-name">--------</a></div><a class="date-and-source">3分钟前 · 来自雪球</a></div><div class="timeline__item__bd"><div class="content content--description">------------------------------------------------------------------------------------------------------------------------------------------------------</div></div><div class="timeline__item__ft"><div class="timeline__item__control"></div></div></div></div></div></div></div><home-timeline></home-timeline></div><div class="home__col--rt"><ad-right-aside pos-id="right_top" position-id="10323020601" platform-id="0x05" page-id="0x05" mod-id="01"></ad-right-aside><side-nav></side-nav><today-topic-list></today-topic-list><talks-list></talks-list><stock-hot-list></stock-hot-list><most-profitable-list></most-profitable-list><recommend-user :classname="['recommend-user__container']"></recommend-user><div class="home__reward__entry board"><h3>悬赏提问<a class="more" href="/law/prize/faq" target="_blank">常见问题</a></h3><div class="home__reward__entry__bd"><p>有问题不知道问谁?<br>试试悬赏提问</p><a class="button button-outline-gold" href="">发起悬赏</a></div></div><div class="other-service__container board"><h3>其他服务</h3><table class="footer-right_list"><td><a href="/broker/open/pingan?from=foot" target="_blank">开户指南</a></td><td><a href="/operation/guide" target="_blank">使用指南</a></td><td><a href="/about/faq" target="_blank">常见问题</a></td><td><a href="/verify/index" target="_blank">认证申请</a></td></table><p>其他建议反馈欢迎点击&nbsp;<a href="https://xueqiu.com/hashtag/I-aIkee7membqueQg-aPkOW7uuiuriM=" target="_blank">#我给雪球提建议#</a></p><p>如受到欺诈信息骚扰请至&nbsp;<a href="https://xueqiu.com/hashtag/I-mYsuiviOmql-S4vuaKpeS4k-WMuiM=" target="_blank">#防诈骗举报专区#</a></p></div><side-report-entrance></side-report-entrance><ad-right-aside pos-id="right_bottom" position-id="10323020602" platform-id="0x05" page-id="0x05" mod-id="01"></ad-right-aside></div></div></div><page-footer></page-footer><div class="modals dimmer"><modals></modals><confirms></confirms><alert></alert><alert-pay-success></alert-pay-success></div><div class="toast"><i class="iconfont">&#xe630;</i><i class="iconfont error-icon">&#xe631;</i><i class="iconfont warn-icon">&#xe632;</i><i class="iconfont loading-icon">&#xe64a;</i><span></span></div><tooltip></tooltip><top :target="'.user__col--middle'" :reference="'.home__col--rt'"></top><upgrade-tip></upgrade-tip><widget-download-app></widget-download-app></div><script src="//assets.imedao.com/ugc/js/jquery-3-bf5be4b4bc.1.1.js" rel="preload" as="script"></script><script src="//assets.imedao.com/ugc/js/polyfill-7-1a461e6e65.4.4.js" rel="preload" as="script"></script><script src="//assets.imedao.com/ugc/js/raven-f24a7020e8.min.js"></script><script>Raven.config('https://9f2e0acfac5e4a3ba8fcc2cf22b69b11@sentry.xueqiu.com/29', {sampleRate: 0.2}).install()
Raven.setUserContext({
  id: SNOWMAN_USER.id || -1,
  username: SNOWMAN_USER.screen_name || '',
  ip_address: '123.149.30.241'
});</script><script src="//assets.imedao.com/ugc/js/medium-editor-bd85ca59df.js"></script><script src="//assets.imedao.com/ugc/js/page-b3a1971449.js"></script><script>var _hmt = _hmt || [];
(function() {
  var hm = document.createElement("script");
  hm.src = "//hm.baidu.com/hm.js?1db88642e346389874251b5a1eded6e3";
  var s = document.getElementsByTagName("script")[0];
  s.parentNode.insertBefore(hm, s);
})();</script><script src="//assets.imedao.com/ugc/js/apm-4-df4081db31.1.js"></script><script src="//assets.imedao.com/ugc/js/env-48c33adc63.js"></script><script>if (window.localStorage.getItem('snb-logout-flag') && window.needAutoOpenLoginModal()) {
  $.subscribe('vue-ready', function(e, bus){
    if (!window.SNOWMAN_LOGIN && !JSON.parse(window.localStorage.getItem('snb-modal-login-show') || 'false')) {
      bus.$emit('modal-login', 1, {
        trigger: 'quit'
      });
    }
  });
  window.localStorage.setItem('snb-logout-flag', false)
};
if(window && window.elasticApm){
  var apm = window.elasticApm.init({
    serviceName: 'snowman-web',
    serverUrl: 'https://open.xueqiu.com/es-apm',
    transactionSampleRate: 0.1,
  })
  var apmTransactionName = location.pathname;

  if (apmTransactionName.match(/^\/S\//i)) {
    //- 个股页
    apmTransactionName = 'stock pages';
  } else if (apmTransactionName.match(/^\/$/)){
    //- 首页
    apmTransactionName = 'home page';
  } else if (apmTransactionName.match(/^\/[0-9]{10}\/[0-9]{8}/)) {
    //- 文章页
    apmTransactionName = 'article pages';
  } else if (apmTransactionName.match(/^\/(u\/|[0-9]{10})/)) {
    //- 个人页
    apmTransactionName = 'profile pages';
  } else {
    //- 其他页面
    apmTransactionName = 'other pages';
  }
  apm.setInitialPageLoadName(apmTransactionName);
  if (window.SNOWMAN_LOGIN) {
    apm.setUserContext({
      id: window.SNOWMAN_USER.id,
      username: window.SNOWMAN_USER.screen_name
    })
  }
}</script><script src="//assets.imedao.com/ugc/js/vue-common-6118e8a4ff.js"></script><script src="//assets.imedao.com/ugc/js/vue-home-1033b4aeaf.js"></script><script src="//assets.imedao.com/ugc/js/underscore-dbf542885f.js"></script><script src="//assets.imedao.com/ugc/js/backbone-9c5159b315.js"></script><script src="//assets.imedao.com/ugc/js/im-8fcce4c918.js"></script><script>$.ajax('/snowman/f2e/statistics?from=snowman');
// 进入埋点
try {
  $(document).trigger('analytics', {
    page: 500,
    type: 0
  });
} catch (error) {
}</script></body></html>

python代码:

python">import requests

headers = {
    'User-Agent': 'Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/103.0.0.0 Safari/537.36',
    'Host': 'xueqiu.com',
    'Referer': 'https://xueqiu.com/',
    'Cookie': 'cookiesu=411700482632361; device_id=29270365565b168f60a35e878597d7a3; s=am11lj11rm; __utmz=1.1700482786.1.1.utmcsr=(direct)|utmccn=(direct)|utmcmd=(none);acw_tc=2760825d17051362899667830e47536d0c8ad57ed00a73bbefb6c4b513e5ee; Hm_lvt_1db88642e346389874251b5a1eded6e3=1705136319; u=6608945068; snbim_minify=true; bid=4e57f453e306324b754929f18b8e3530_lrbu8ecs; __utma=1.2074947920.1700482786.1700482786.1705136477.2; __utmc=1; __utmt=1; remember=1; xq_a_token=34346c84cf49fc520a74b727e01a599d017d6960;xqat=34346c84cf49fc520a74b727e01a599d017d6960;xq_id_token=eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiJ9.eyJ1aWQiOjY2MDg5NDUwNjgsImlzcyI6InVjIiwiZXhwIjoxNzA3NzI4NjM2LCJjdG0iOjE3MDUxMzY2MzYzNDYsImNpZCI6ImQ5ZDBuNEFadXAifQ.SEcNWjvbF4GwO-dVZY-AJxXJ3TBTqGqV_Xac5S034vvnx5cYicltppGpv4vfXjzFSdgMYhby1cLR0CzxAKjtbItJre1ZIz_814TC8v4524L0LajorvNNujiF_9FccrZ6Z1PT-GZ6l1Q8n22s3KxafEGM64Hr2VK6KoWJpwTzvSywD3SUwUoNTuHLV0Pm2fn9esdB9mcnCTZkRA3HiRlVh7XcBY7K8T9frLbMTG9ssXZ1JzDjaGwB07fAxRLteblFWWbelk8db77ZK066q7OlZ1bVexVBHdqkXu_vh2Snfud9f8x3_Sjj7a_byBgL5A5L5MJWpmrmZHqJlyKCXiLGQ; xq_r_token=2f87a9dd64211e24c310cb58f2169f0f6e91723c; xq_is_login=1; Hm_lpvt_1db88642e346389874251b5a1eded6e3=1705136778; __utmb=1.9.10.1705136477; is_overseas=0',
}
url = 'https://xueqiu.com/statuses/hot/listV2.json?since_id=-1&max_id=366242&size=15'

response = requests.get(url, headers=headers)
print(response.json())

运行结果:有数据

{'next_max_id': -1, 'items': [], 'next_id': -1}

此实例只是为了验证,某些网站反爬必须需要携带cookies。


http://www.niftyadmin.cn/n/5322628.html

相关文章

在CMake中自定义宏 add_definitions(-DDEBUG)

hehedalinux:~/Linux/loveDBTeacher-v6$ tree . ├── CMakeLists.txt └── test.c0 directories, 2 files hehedalinux:~/Linux/loveDBTeacher-v6$ test.c #include <stdio.h> #define NUMBER 3int main() {int a 10; #ifdef DEBUGprintf("我是一个程序猿,我…

OpenAI ChatGPT-4开发笔记2024-06:最简Embedding

Embedding embedding直译是&#xff1a;嵌入。和实际意思有些差距。其实就是把文本转换为向量表示的过程。用“向量化”更直接&#xff0c;但这又和tensor有点儿混。它是变成向量的一个过程。 embedding 的应用领域&#xff1a; 文本分类&#xff1a; 将文本嵌入转换为向量后…

监督学习 - 梯度提升机(Gradient Boosting Machines,GBM)

什么是机器学习 梯度提升机&#xff08;Gradient Boosting Machines&#xff0c;GBM&#xff09;是一种集成学习方法&#xff0c;通过将多个弱学习器&#xff08;通常是决策树&#xff09;组合成一个强学习器来提高模型的性能。GBM的训练过程是通过迭代&#xff0c;每一步都根…

git如何仅合并项目下的某个文件夹

前言 假设目前在做的一个项目名为demoproject&#xff0c;目前版本为v1.0.0&#xff0c;标准开发都是在demoproject上进行的&#xff0c;但是有一个客户需要做定制化&#xff0c;需要将项目中的一些信息改为客户的&#xff0c;于是基于demoproject拉了一个v1.0.0-custom&#…

【python入门】day26: 模拟高铁售票系统

界面 代码 #-*- coding:utf-8 -*- import prettytable as pt#---------导入漂亮表格 import os.path filename ticket.txt#更新座位状态 def update(row_num):#------更新购票状态with open(filename,w,encodingutf-8) as wfile:for i in range(row_num):lst1 [f{i1},有票,有…

【Python_PySide6学习笔记(二十六)】基于PySide6实现自定义标题栏类tQTitleBar,实现窗体的无边框美化

基于PySide6实现自定义标题栏类tQTitleBar,实现窗体的无边框美化 基于PySide6实现自定义标题栏类tQTitleBar,实现窗体的无边框美化前言一、实现思路1、继承自 QWidget 类,创建控件并添加2、编写窗体最大化、最小化、关闭、最大化后恢复的方法3、重写鼠标点击、移动、释放、双…

如何解决NAND系统性能问题?-- NAND接口分类

三、NAND接口 NAND闪存接口是连接主机控制器与NAND存储芯片的通信桥梁&#xff0c;负责命令、地址和数据的传输。典型的NAND闪存接口包括一组I/O线&#xff08;通常为8条或更多&#xff09;用于数据传输&#xff0c;以及若干控制信号线。 基本接口信号&#xff1a; Chip Enable…

Linux|服务器|简单记录备忘VMware虚拟机开启桌面失败报错:VMware: No 3D enabled (0, Success).的解决

一&#xff0c; VMware虚拟机 Linux操作系统&#xff0c;centos7版本&#xff0c;安装完桌面后&#xff0c;执行startx 命令后 &#xff0c;报错&#xff1a;VMware: No 3D enabled (0, Success). 桌面没有启动成功 完整日志输出如下&#xff1a; [rootnode4 ~]# startx x…