《C# 爬虫 破境之道》:第一境 爬虫原理 — 第三节:WebResponse
时间:2020-01-03
本文章向大家介绍《C# 爬虫 破境之道》:第一境 爬虫原理 — 第三节:WebResponse,主要包括《C# 爬虫 破境之道》:第一境 爬虫原理 — 第三节:WebResponse使用实例、应用技巧、基本知识点总结和需要注意事项,具有一定的参考价值,需要的朋友可以参考一下。
第二节中,我们介绍了WebRequest,它可以帮助我们发送一个请求,不过正所谓“来而不往非礼也”,对方收到我们的请求,不给点回复,貌似不太合适(不过,还真有脸皮厚的:P)。
接下来,就重点研究一下,我们收到的回复,是个什么样的东东
[Code 1.3.1]
1 // 2 // Summary: 3 // Provides a response from a Uniform Resource Identifier (URI). This is an abstract 4 // class. 5 public abstract class WebResponse : MarshalByRefObject, ISerializable, IDisposable 6 { 7 // 8 // Summary: 9 // Initializes a new instance of the System.Net.WebResponse class. 10 protected WebResponse(); 11 // 12 // Summary: 13 // Initializes a new instance of the System.Net.WebResponse class from the specified 14 // instances of the System.Runtime.Serialization.SerializationInfo and System.Runtime.Serialization.StreamingContext 15 // classes. 16 // 17 // Parameters: 18 // serializationInfo: 19 // An instance of the System.Runtime.Serialization.SerializationInfo class that 20 // contains the information required to serialize the new System.Net.WebRequest 21 // instance. 22 // 23 // streamingContext: 24 // An instance of the System.Runtime.Serialization.StreamingContext class that indicates 25 // the source of the serialized stream that is associated with the new System.Net.WebRequest 26 // instance. 27 // 28 // Exceptions: 29 // T:System.NotSupportedException: 30 // Any attempt is made to access the constructor, when the constructor is not overridden 31 // in a descendant class. 32 protected WebResponse(SerializationInfo serializationInfo, StreamingContext streamingContext); 33 34 // 35 // Summary: 36 // Gets a System.Boolean value that indicates whether this response was obtained 37 // from the cache. 38 // 39 // Returns: 40 // true if the response was taken from the cache; otherwise, false. 41 public virtual bool IsFromCache { get; } 42 // 43 // Summary: 44 // Gets a System.Boolean value that indicates whether mutual authentication occurred. 45 // 46 // Returns: 47 // true if both client and server were authenticated; otherwise, false. 48 public virtual bool IsMutuallyAuthenticated { get; } 49 // 50 // Summary: 51 // When overridden in a descendant class, gets or sets the content length of data 52 // being received. 53 // 54 // Returns: 55 // The number of bytes returned from the Internet resource. 56 // 57 // Exceptions: 58 // T:System.NotSupportedException: 59 // Any attempt is made to get or set the property, when the property is not overridden 60 // in a descendant class. 61 public virtual long ContentLength { get; set; } 62 // 63 // Summary: 64 // When overridden in a derived class, gets or sets the content type of the data 65 // being received. 66 // 67 // Returns: 68 // A string that contains the content type of the response. 69 // 70 // Exceptions: 71 // T:System.NotSupportedException: 72 // Any attempt is made to get or set the property, when the property is not overridden 73 // in a descendant class. 74 public virtual string ContentType { get; set; } 75 // 76 // Summary: 77 // When overridden in a derived class, gets the URI of the Internet resource that 78 // actually responded to the request. 79 // 80 // Returns: 81 // An instance of the System.Uri class that contains the URI of the Internet resource 82 // that actually responded to the request. 83 // 84 // Exceptions: 85 // T:System.NotSupportedException: 86 // Any attempt is made to get or set the property, when the property is not overridden 87 // in a descendant class. 88 public virtual Uri ResponseUri { get; } 89 // 90 // Summary: 91 // When overridden in a derived class, gets a collection of header name-value pairs 92 // associated with this request. 93 // 94 // Returns: 95 // An instance of the System.Net.WebHeaderCollection class that contains header 96 // values associated with this response. 97 // 98 // Exceptions: 99 // T:System.NotSupportedException: 100 // Any attempt is made to get or set the property, when the property is not overridden 101 // in a descendant class. 102 public virtual WebHeaderCollection Headers { get; } 103 // 104 // Summary: 105 // Gets a value that indicates if headers are supported. 106 // 107 // Returns: 108 // Returns System.Boolean. true if headers are supported; otherwise, false. 109 public virtual bool SupportsHeaders { get; } 110 111 // 112 // Summary: 113 // When overridden by a descendant class, closes the response stream. 114 // 115 // Exceptions: 116 // T:System.NotSupportedException: 117 // Any attempt is made to access the method, when the method is not overridden in 118 // a descendant class. 119 public virtual void Close(); 120 // 121 // Summary: 122 // Releases the unmanaged resources used by the System.Net.WebResponse object. 123 public void Dispose(); 124 // 125 // Summary: 126 // When overridden in a descendant class, returns the data stream from the Internet 127 // resource. 128 // 129 // Returns: 130 // An instance of the System.IO.Stream class for reading data from the Internet 131 // resource. 132 // 133 // Exceptions: 134 // T:System.NotSupportedException: 135 // Any attempt is made to access the method, when the method is not overridden in 136 // a descendant class. 137 public virtual Stream GetResponseStream(); 138 // 139 // Summary: 140 // Releases the unmanaged resources used by the System.Net.WebResponse object, and 141 // optionally disposes of the managed resources. 142 // 143 // Parameters: 144 // disposing: 145 // true to release both managed and unmanaged resources; false to releases only 146 // unmanaged resources. 147 protected virtual void Dispose(bool disposing); 148 // 149 // Summary: 150 // Populates a System.Runtime.Serialization.SerializationInfo with the data that 151 // is needed to serialize the target object. 152 // 153 // Parameters: 154 // serializationInfo: 155 // The System.Runtime.Serialization.SerializationInfo to populate with data. 156 // 157 // streamingContext: 158 // A System.Runtime.Serialization.StreamingContext that specifies the destination 159 // for this serialization. 160 protected virtual void GetObjectData(SerializationInfo serializationInfo, StreamingContext streamingContext); 161 }
相比较WebRequest而言,WebResponse简单许多,这里就不再分解片断讲解了。有事儿说事儿,不要以貌取人:D
首先,与WebRequest的类似,它还是抽象类,两个构造函数的结构,一票子虚属性,大部分都是只读的,和一票子虚方法。
其次,与WebRequest不同,它继承了System.IDisposable接口,提醒我们啊,用完了,要释放啊~~~~~~
几个属性
[Code 1.3.2]
1 /// <summary> 2 /// 只读,指示这个回复是不是从缓存中获取的 3 /// </summary> 4 public virtual bool IsFromCache { get; } 5 /// <summary> 6 /// 只读,指示是否发生相互认证 7 /// </summary> 8 public virtual bool IsMutuallyAuthenticated { get; } 9 /// <summary> 10 /// 指示收到的数据长度 11 /// </summary> 12 public virtual long ContentLength { get; set; } 13 /// <summary> 14 /// 指示收到的内容类型 15 /// </summary> 16 public virtual string ContentType { get; set; } 17 /// <summary> 18 /// 只读,指示收到的实际资源URI 19 /// </summary> 20 public virtual Uri ResponseUri { get; } 21 /// <summary> 22 /// 只读,Header集合,HTTP协议中的重点 23 /// </summary> 24 public virtual WebHeaderCollection Headers { get; } 25 /// <summary> 26 /// 只读,指示是否支持Header集合 27 /// </summary> 28 public virtual bool SupportsHeaders { get; }
重要的属性有Headers、ContentType、ContentLength。对于分析数据都是举足轻重的。
几个方法
[Code 1.3.3]
1 /// <summary> 2 /// 获得目标资源的数据流 3 /// </summary> 4 public virtual Stream GetResponseStream(); 5 /// <summary> 6 /// 关闭Response流 7 /// </summary> 8 public virtual void Close(); 9 /// <summary> 10 /// 释放WebResponse对象 11 /// </summary> 12 public void Dispose(); 13 /// <summary> 14 /// 释放WebResponse对象 15 /// </summary> 16 protected virtual void Dispose(bool disposing); 17 /// <summary> 18 /// 使用序列化目标对象所需的数据填充System.Runtime.Serialization.SerializationInfo 19 /// </summary> 20 protected virtual void GetObjectData(SerializationInfo serializationInfo, StreamingContext streamingContext);
重要的方法有GetResponseStream、Close。对于获取数据至关重要。
总而言之,WebResponse还是比较简单的,给我们留下的坑不多。与WebRequest的讲解一样,干嚼无味,还是在实例中体会个中滋味吧:P
原文地址:https://www.cnblogs.com/mikecheers/p/12144431.html
- python爬取链家租房之获取房屋页面的详细信息(房名,地址,房价,面积,url)
- 信用卡“坏账”客户分析(二)
- Mac上提升python运算速度-PyPy初体验
- python处理json数据(复杂的json转化成嵌套字典并处理)
- Pointer-network理论及tensorflow实战
- python3 log文件处理获取某天需要的数据
- 实战深度强化学习DQN-理论和实践
- Python 工匠:编写条件分支代码的技巧
- python3 下 Zabbix监控调用graph.get并且下载监控图
- 用R语言复盘美国总统大选结果~
- 最新Apache Spark平台的NLP库,助你轻松搞定自然语言处理任务
- 使用Seq2Seq+attention实现简单的Chatbot
- R语言可视化——用ggplot构造期待已久的雷达图
- 【实战】最新Deep Learning with Keras图书加代码,教你从零开发一个复杂深度学习模型(附下载)
- JavaScript 教程
- JavaScript 编辑工具
- JavaScript 与HTML
- JavaScript 与Java
- JavaScript 数据结构
- JavaScript 基本数据类型
- JavaScript 特殊数据类型
- JavaScript 运算符
- JavaScript typeof 运算符
- JavaScript 表达式
- JavaScript 类型转换
- JavaScript 基本语法
- JavaScript 注释
- Javascript 基本处理流程
- Javascript 选择结构
- Javascript if 语句
- Javascript if 语句的嵌套
- Javascript switch 语句
- Javascript 循环结构
- Javascript 循环结构实例
- Javascript 跳转语句
- Javascript 控制语句总结
- Javascript 函数介绍
- Javascript 函数的定义
- Javascript 函数调用
- Javascript 几种特殊的函数
- JavaScript 内置函数简介
- Javascript eval() 函数
- Javascript isFinite() 函数
- Javascript isNaN() 函数
- parseInt() 与 parseFloat()
- escape() 与 unescape()
- Javascript 字符串介绍
- Javascript length属性
- javascript 字符串函数
- Javascript 日期对象简介
- Javascript 日期对象用途
- Date 对象属性和方法
- Javascript 数组是什么
- Javascript 创建数组
- Javascript 数组赋值与取值
- Javascript 数组属性和方法
- ROS2机器人笔记20-08-18
- C sharp实例:华盾武器门数据接收和解析
- JsonPath实践(三)
- 有序链表转换二叉搜索树
- 128. 最长连续序列
- 【DB笔试面试859】在Oracle中,内核参数kernel.shmall、kernel.shmall等分别代表什么含义?
- Nginx配置SSL证书
- java_Object类、日期时间类、System类、包装类
- [周末往期回顾]UNDO_TABLESPACE参数
- WordPress固定链接后404解决方法
- [Oracle故障处理]记一次INST_DRTLD_MISMATCH导致的version count过多的问题
- 查表法实现十进制转化成其他进制
- ArrayList源码分析
- java_String类、StringBuilder类、Arrays类、Math类的使用
- 抽象类与接口