Section 14.3 Plan 2: Get a soup from a URL

Subsection 14.3.1 Plan 2: Example

The first step in web scraping is getting information from a webpage.

To use the BeautifulSoup web scraping library, we have to put the webpage into something called a soup.

Here is the code for getting a soup from the Cottage Inn location page.

# Load libraries for web scraping
from bs4 import BeautifulSoup
import requests
# Get a soup from a URL
url = 'https://web.archive.org/web/20200427175705/https://cottageinn.com/pick-a-location/'
r = requests.get(url)
soup = BeautifulSoup(r.content, 'html.parser')

Subsection 14.3.2 Plan 2: When to use this plan

Use this plan when you want to scrape one webpage.

Subsection 14.3.3 Plan 2: How to use this plan

Replace the URL with the URL of the website you want to scrape.

A URL is a web address, like you see in your web browser.

It should be complete (starting with http:// or https://).

In this plan, a URL should be surrounded by quotes (' ').

Figure 14.3.1. Copying a URL from the Cottage Inn location page

Subsection 14.3.4 Plan 2: Exercises

Activity 14.3.1.

If you wanted to get a soup from the MDen homepage instead of the Cottage Inn location page, which part(s) of the code below would you change? Click on those part(s) of the code.

Check out the example of this plan above to identify the area that should be changed.

# Load libraries for web scraping
from bs4 import BeautifulSoup
import requests

# Get a soup from a URL
url =’https://web.archive.org/web/20200427175705/https://cottageinn.com/pick-a-location/’
r = requests.get(url)
soup = BeautifulSoup(r.content, ’html.parser’)

Activity 14.3.2.

Note 14.3.2.

Click here to go back to the Cottage Inn example

You have attempted of activities on this page.

Prev Top Next