Section 14.6 Write Code Questions - Parsons for Select

Activity 14.6.1.

Create a function called cur_stud_link(soup) that takes a soup object and returns the URL that links to Current Students in the Navigation Bar. The soup object is created from the file UMich-8-16-2026.html which is a saved version of “https://umich.edu/” from 2026. Click on the following link to view the source:link. For example, cur_stud_link(soup) should return 'https://umich.edu/web/20260808043645/https://umich.edu/current-students/'.

import requests
from bs4 import BeautifulSoup
---
def cur_stud_link(soup):
---
    text = soup.find('a', string="Current Students")
---
    text = soup.find_all('a', string="Current Students")   #paired
---
    info = text.get('href')
---
    info = text.get(href)  #paired
---
    all_info = "https://umich.edu" + info
---
    return all_info

Activity 14.6.2.

Create a function called headings(soup) that takes a soup object and returns a list of the text from all h2 tags in the soup. The soup object is created from the file MOOC-Wikipedia-8-20-2026.html which is a saved version of “https://en.wikipedia.org/wiki/Massive_open_online_course” from 2026. Click on the following link to view the source: link For example, headings(soup) should return a list of 14 headings starting with ['Contents', 'History', 'Equity and access', ...].

import requests
from bs4 import BeautifulSoup
---
def headings(soup):
    out_list = []
---
    h_list = soup.find_all("h2")
---
    h_list = soup.find("h2")    #paired
---
    for h_tag in h_list:
---
        out_list.append(h_tag.text)
---
        out_list.append(h_tag)  #paired
---
    return out_list

Activity 14.6.3.

Create a function called coursera_dict(soup) that takes a soup object and returns a dictionary with Country as keys and Percentage as values. The soup object is created from the file ’WikipediaMOOC-Jan-12-2018.html’ which is a saved version of “https://en.wikipedia.org/wiki/Massive_open_online_course” from 2018. Click on the following link to view the source: link For example, coursera_dict(soup) should return a dictionary like: {'United States': '27.7%', 'India': '8.8%', 'Brazil': '5.1%', ...}.

from bs4 import BeautifulSoup
import requests
---
def coursera_dict(soup):
    coursera_enrollees = {}
---
    tables = soup.find_all('table', class_='wikitable')
---
    tables = soup.find('table', class_='wikitable')  #paired
---
    rows = tables[1].find_all('tr')
---
    for row in rows[1:]:
---
          both_columns = row.find_all("td")
---
          key = both_columns[0].text
          value = both_columns[1].text.rstrip("\n")
---
          coursera_enrollees[key] = value
---
    return coursera_enrollees

Activity 14.6.4.

Create a function called keywords(soup) that takes a soup object and returns a list of Python keywords. The soup object is created from the file PythonKeywords.html which is a saved version of “https://www.w3schools.com/python/python_ref_keywords.asp” from 2026. Click on the following link to view the source: link. For example, keywords(soup) should return a list that starts with: ['and', 'as', ...].

import requests
from bs4 import BeautifulSoup
---
def keywords(soup):
    keywords = []
---
    table = soup.find('table', class_="ws-table-all")
---
    table = soup.find(table) #paired
---
    rows = table.find_all('tr')
---
    for row in rows[1:]:
---
    for row in rows:  #paired
---
        data = row.find("td")
---
        keyword = data.find('a').text
---
        keywords.append(keyword)
---
    return keywords

Activity 14.6.5.

Create a function called restaurants_list(file) that takes in a file name as file, uses BeautifulSoup to web scrape the 13 best restaurants in Ann Arbor as of 2026, For example, restaurants_list('AnnArborBestRestaurants2026.html') should return ['Sava’s', 'Knight’s Steakhouse', 'Mani Osteria and Bar', ...]. The current url is “https://detroit.eater.com/maps/best-ann-arbor-restaurants”. We are using a saved version of that site from Aug 1 2026. Click on the following link to view the source link.

from bs4 import BeautifulSoup
import requests
---
def restaurants_list(file):
    r_list = []
---
    fh = open(file)
    text = fh.read()
    fh.close()
---
    soup = BeautifulSoup(text, 'html.parser')
---
    soup = BeautifulSoup(text, 'xml')  #paired
---
    rest_list = soup.find_all("h2", class_="hkfm3h5")
---
    rest_list = soup.find('h2', class_="hkfm3h5")  #paired
---
    for rest in rest_list:
---
        r_list.append(rest.text.strip())
---
    return r_list

Activity 14.6.6.

Create a function called bsi_list that takes in a parameter url, uses BeautifulSoup to web scrape the section names and subsection names under Programs Bachelor of Science in Information, and returns a list of the section names and subsection names under Programs Bachelor of Science in Information. For example, bsi_list('https://www.si.umich.edu/programs/bachelor-science-information') should return ['How do I apply?', 'Current U-M students', 'Transfer students', ...]. Since we are scraping a live site the results may be different.

import requests
from bs4 import BeautifulSoup
---
def bsi_list(url):
    list1 = []
---
    r = requests.get(url)
---
    soup = BeautifulSoup(r.text, 'html.parser')
---
    sections = soup.find_all('a', href = re.compile("/programs/bachelor-science-information/"))
---
    for section in sections[:-2]:
---
        list1.append(section.text.strip("\n").strip())
---
        list1.append(section.strip("\n").strip())  #paired
---
    return list1

Activity 14.6.7.

Create a function called bsi_dict that takes in a parameter url, uses BeautifulSoup to web scrape and create a list of the section names and subsection names under Programs Bachelor of Science in Information, and creates another list that contains the full urls of the named sections and subsections. Then, return a dictionary that contains the names as keys and their full urls as values. For example, bsi_dict('https://www.si.umich.edu/programs/bachelor-science-information') should return {'How do I apply?': 'https://www.si.umich.edu/.../how-do-i-apply', ...}. Since we are scraping a live site the results may be different.

from bs4 import BeautifulSoup
import requests
import re
---
def bsi_dict(url):
    bsi_dict = {}
---
    r = requests.get(url)
    soup = BeautifulSoup(r.text, 'html.parser')
---
    r = requests.get(url)
    soup = BeautifulSoup(r.string, 'html.parser')  #paired
---
    sections = soup.find_all('a', href = re.compile("/programs/bachelor-science-information/"))
---
    sections = soup.find_all('link', href = re.compile("/programs/bachelor-science-information/"))  #paired
---
    for section in sections[:-2]:
---
        key = section.text.strip()
        value = section.get('href')
---
        bsi_dict[bsi_list[key] = value
---
    return bsi_dict

Activity 14.6.8.

Create a function called envelope_address that takes in a parameter url and uses BeautifulSoup to web scrape and return the address text in the footer as a list. For example, envelope_address('https://www.si.umich.edu/programs/bachelor-science-information') should return a list like: ['School of Information', 'University of Michigan', ...]. Since we are scraping a live site the results may be different.

import requests
from bs4 import BeautifulSoup
---
def envelope_address(url):
    out_list = []
---
    r = requests.get(url)
---
    soup = BeautifulSoup(r.text, 'html.parser')
---
    address = soup.find('div', class_ = 'footer--address').text.strip()
---
    address = soup.find('div', class = 'footer--address').text.strip()  #paired
---
    add_list = address.split("\n")
---
    add_list = address.split()  #paired
---
    for add_line in add_list:
---
        out_list.append(add_line.strip())
---
        add_list.append(add_line.strip())  #paired
---
    return out_list

Activity 14.6.9.

Create a function called name_email that takes in a parameter url, uses BeautifulSoup to web scrape the names and email addresses, and returns a dictionary with the names as keys and the email addresses as values. Use the string replace method while web scraping. For example, name_email('https://www.si.umich.edu/people/directory/faculty/e') should return {'Paul Edwards': '[email protected]', 'Ron Eglash': '[email protected]', ...}. Since we are scraping a live site the results may be different.

from bs4 import BeautifulSoup
import requests
---
import re  #paired
---
def name_email(url):
    names_list = []
    emails_list = []
    name_email_dict = {}
---
    r = requests.get(url)
    soup = BeautifulSoup(r.text, 'html.parser')
---
    r = requests.get(url)
    soup = BeautifulSoup(r.string, 'html.parser')  #paired
---
    names = soup.find_all(class_="research-person-profile__name")
---
    names = soup.find_all(class="research-person-profile__name")  #paired
---
    emails = soup.find_all("a", target="_blank")
---
    emails = soup.find_all("link", target="_blank")  #paired
---
    for name in names:
---
        profile_name = name.text.strip().replace('\n \n ', '')
        names_list.append(profile_name)
---
    for email in emails:
---
        profile_email = email.get("href")
        profile_email = profile_email.replace('mailto:', '')
        emails_list.append(profile_email)
---
    for i in range(len(names_list)):
        name_email_dict[names_list[i]] = emails_list[i]
---
    return name_email_dict

Activity 14.6.10.

Create a function called program_email that takes in a parameter url, uses BeautifulSoup to web scrape the program names under Email Addresses and Admissions (BSI program, MSI program, etc.) and their associated email addresses, and returns a dictionary with the program names as keys and the email addresses as values. For example, program_email('https://www.si.umich.edu/about-umsi/contact-us') should return something like {'BSI program': '[email protected]', 'MSI program': '[email protected]', ...}. Since we are scraping a live site the results may be different.

import requests
from bs4 import BeautifulSoup
---
def program_email(url):
    program_email_dict = {}
---
    r = requests.get(url)
---
    soup = BeautifulSoup(r.text, 'html.parser')
---
    programs_section = soup.find("ul", dir="ltr")
---
    programs = programs_section.find_all("li")
---
    programs = programs_section.find("li")  #paired
---
    for program in programs:
---
        split_name_email = program.text.split(":")
---
        program_name = split_name_email[0]
---
        program_email = split_name_email[1].replace("\xa0", "")
---
        program_email = split_name_email[0].replace("\xa0", "")  #paired
---
        program_email_dict[program_name] = program_email
---
    return program_email_dict

You have attempted of activities on this page.

Prev Top Next