Web Scraping in Python

These are my notes on Web Scraping in Python

Libraries

Snippets

Scrape a web page behind a login

from requests_html import HTMLSession

session = HTMLSession()

login_page = session.post(
  "https://example.com/login.php",
  data={
    "username": "myles",
    "password": "areallygoodpassword"
  }
)

if not login_page.ok:
  raise Exception

secret_page = session.get(
  "https://example.com/admin/index.php",
  cookies=login_page.cookies
)

if not secret_page.pk:
  raise Exception

results matching ""

    powered by

    No results matching ""