requests几个常用方法源码

  1. def request(method, url, **kwargs):
  2.     """Constructs and sends a :class:`Request <Request>`.
  4.     :param method: method for the new :class:`Request` object: ``GET``, ``OPTIONS``, ``HEAD``, ``POST``, ``PUT``, ``PATCH``, or ``DELETE``.
  5.     :param url: URL for the new :class:`Request` object.
  6.     :param params: (optional) Dictionary, list of tuples or bytes to send
  7.         in the query string for the :class:`Request`.
  8.     :param data: (optional) Dictionary, list of tuples, bytes, or file-like
  9.         object to send in the body of the :class:`Request`.
  10.     :param json: (optional) A JSON serializable Python object to send in the body of the :class:`Request`.
  11.     :param headers: (optional) Dictionary of HTTP Headers to send with the :class:`Request`.
  12.     :param cookies: (optional) Dict or CookieJar object to send with the :class:`Request`.
  13.     :param files: (optional) Dictionary of ``'name': file-like-objects`` (or ``{'name': file-tuple}``) for multipart encoding upload.
  14.         ``file-tuple`` can be a 2-tuple ``('filename', fileobj)``, 3-tuple ``('filename', fileobj, 'content_type')``
  15.         or a 4-tuple ``('filename', fileobj, 'content_type', custom_headers)``, where ``'content-type'`` is a string
  16.         defining the content type of the given file and ``custom_headers`` a dict-like object containing additional headers
  17.         to add for the file.
  18.     :param auth: (optional) Auth tuple to enable Basic/Digest/Custom HTTP Auth.
  19.     :param timeout: (optional) How many seconds to wait for the server to send data
  20.         before giving up, as a float, or a :ref:`(connect timeout, read
  21.         timeout) <timeouts>` tuple.
  22.     :type timeout: float or tuple
  23.     :param allow_redirects: (optional) Boolean. Enable/disable GET/OPTIONS/POST/PUT/PATCH/DELETE/HEAD redirection. Defaults to ``True``.
  24.     :type allow_redirects: bool
  25.     :param proxies: (optional) Dictionary mapping protocol to the URL of the proxy.
  26.     :param verify: (optional) Either a boolean, in which case it controls whether we verify
  27.             the server's TLS certificate, or a string, in which case it must be a path
  28.             to a CA bundle to use. Defaults to ``True``.
  29.     :param stream: (optional) if ``False``, the response content will be immediately downloaded.
  30.     :param cert: (optional) if String, path to ssl client cert file (.pem). If Tuple, ('cert', 'key') pair.
  31.     :return: :class:`Response <Response>` object
  32.     :rtype: requests.Response
  34.     Usage::
  36.       >>> import requests
  37.       >>> req = requests.request('GET', 'https://httpbin.org/get')
  38.       >>> req
  39.       <Response [200]>
  40.     """
  42.     # By using the 'with' statement we are sure the session is closed, thus we
  43.     # avoid leaving sockets open which can trigger a ResourceWarning in some
  44.     # cases, and look like a memory leak in others.
  45.     with sessions.Session() as session:
  46.         return session.request(method=method, url=url, **kwargs)
  49. def get(url, params=None, **kwargs):
  50.     r"""Sends a GET request.
  52.     :param url: URL for the new :class:`Request` object.
  53.     :param params: (optional) Dictionary, list of tuples or bytes to send
  54.         in the query string for the :class:`Request`.
  55.     :param \*\*kwargs: Optional arguments that ``request`` takes.
  56.     :return: :class:`Response <Response>` object
  57.     :rtype: requests.Response
  58.     """
  60.     kwargs.setdefault('allow_redirects', True)
  61.     return request('get', url, params=params, **kwargs)
  64. def options(url, **kwargs):
  65.     r"""Sends an OPTIONS request.
  67.     :param url: URL for the new :class:`Request` object.
  68.     :param \*\*kwargs: Optional arguments that ``request`` takes.
  69.     :return: :class:`Response <Response>` object
  70.     :rtype: requests.Response
  71.     """
  73.     kwargs.setdefault('allow_redirects', True)
  74.     return request('options', url, **kwargs)
  77. def head(url, **kwargs):
  78.     r"""Sends a HEAD request.
  80.     :param url: URL for the new :class:`Request` object.
  81.     :param \*\*kwargs: Optional arguments that ``request`` takes. If
  82.         `allow_redirects` is not provided, it will be set to `False` (as
  83.         opposed to the default :meth:`request` behavior).
  84.     :return: :class:`Response <Response>` object
  85.     :rtype: requests.Response
  86.     """
  88.     kwargs.setdefault('allow_redirects', False)
  89.     return request('head', url, **kwargs)
  92. def post(url, data=None, json=None, **kwargs):
  93.     r"""Sends a POST request.
  95.     :param url: URL for the new :class:`Request` object.
  96.     :param data: (optional) Dictionary, list of tuples, bytes, or file-like
  97.         object to send in the body of the :class:`Request`.
  98.     :param json: (optional) json data to send in the body of the :class:`Request`.
  99.     :param \*\*kwargs: Optional arguments that ``request`` takes.
  100.     :return: :class:`Response <Response>` object
  101.     :rtype: requests.Response
  102.     """
  104.     return request('post', url, data=data, json=json, **kwargs)
  107. def put(url, data=None, **kwargs):
  108.     r"""Sends a PUT request.
  110.     :param url: URL for the new :class:`Request` object.
  111.     :param data: (optional) Dictionary, list of tuples, bytes, or file-like
  112.         object to send in the body of the :class:`Request`.
  113.     :param json: (optional) json data to send in the body of the :class:`Request`.
  114.     :param \*\*kwargs: Optional arguments that ``request`` takes.
  115.     :return: :class:`Response <Response>` object
  116.     :rtype: requests.Response
  117.     """
  119.     return request('put', url, data=data, **kwargs)
  122. def patch(url, data=None, **kwargs):
  123.     r"""Sends a PATCH request.
  125.     :param url: URL for the new :class:`Request` object.
  126.     :param data: (optional) Dictionary, list of tuples, bytes, or file-like
  127.         object to send in the body of the :class:`Request`.
  128.     :param json: (optional) json data to send in the body of the :class:`Request`.
  129.     :param \*\*kwargs: Optional arguments that ``request`` takes.
  130.     :return: :class:`Response <Response>` object
  131.     :rtype: requests.Response
  132.     """
  134.     return request('patch', url, data=data, **kwargs)
  137. def delete(url, **kwargs):
  138.     r"""Sends a DELETE request.
  140.     :param url: URL for the new :class:`Request` object.
  141.     :param \*\*kwargs: Optional arguments that ``request`` takes.
  142.     :return: :class:`Response <Response>` object
  143.     :rtype: requests.Response
  144.     """
  146.     return request('delete', url, **kwargs)

response

  1. # respone属性
  2. print(respone.text)
  3. print(respone.content)
  5. print(respone.status_code)
  6. print(respone.headers)
  7. print(respone.cookies)
  8. print(respone.cookies.get_dict())
  9. print(respone.cookies.items())
  11. print(respone.url)
  12. print(respone.history)  【】形式  重定向
  14. print(respone.encoding)
  16. #关闭：response.close()
  17. from contextlib import closing
  18. with closing(requests.get('xxx',stream=True)) as response:
  19.     for line in response.iter_content():
  20.     pass

编码问题

  1. res = requests.get(url=url, headers=headers)
  2. res.encoding='gbk'  第一种
  3. res.apparent_encoding  第二种
  1. res = requests.get(url=url, headers=headers)
  2. res.iter_content()     
  4. 用于视频、图片等

解析json

  1. res = requests.get(url=url, headers=headers)
  2. res.json()

SSL证书（了解，基本不用）

  1. res = requests.get(url=url, headers=headers,verify=False)   不验证证书，报错返回200
  1. res = requests.get(url=url, headers=headers,cert={...})   手动加上证书

代理（重点）

  1. res = requests.get(url=url, headers=headers,proxies={'http':'http://localhost:9743',})
  3. 协议+地址和端口号
  5. 代理池：列表里面放ip，随机取值
  7. 高匿代理：后端无法拿到IP
  8. 透明代理：后端可以拿到IP

超时

  1. res = requests.get(url=url, headers=headers,timeout=10)

认证设置（了解，基本不用）

  1. res = requests.get(url=url, headers=headers,auth=())

异常

  1. try:
  2.     r=requests.get('http://www.baidu.com',timeout=0.00001)
  3. except ReadTimeout:
  4.     print('===:')
  5.  except ConnectionError: #网络不通
  6.      print('-----')
  7.  except Timeout:
  8.     print('aaaaa')
  10. except RequestException:
  11.     print('Error')

文件上传

  1. res = requests.post(url=url,files={} )