Web scraping using python code

>>> from bs4 import BeautifulSoup
>>> raw_html = open('contrived.html').read()
>>> html = BeautifulSoup(raw_html, 'html.parser')
>>> for p in html.select('p'):
...     if p['id'] == 'walrus':
...         print(p.text)

'I am the walrus'


import requests
from bs4 import BeautifulSoup

URL = 'https://www.monster.com/jobs/search/?q=Software-Developer&where=Australia'
page = requests.get(URL)

soup = BeautifulSoup(page.content, 'html.parser')

Python web scraping project

def extract_company_from_result(soup):   companies = []  for div in soup.find_all(name=”div”, attrs={“class”:”row”}):    company = div.find_all(name=”span”, attrs={“class”:”company”})    if len(company) > 0:      for b in company:        companies.append(b.text.strip())    else:      sec_try = div.find_all(name=”span”, attrs={“class”:”result-link-source”})        for span in sec_try:          companies.append(span.text.strip()) return(companies) extract_company_from_result(soup)

Source: medium.com

Similar codes

0 vote226 views1 answer
Asn1 parsing error when trying to connect to mongo atlas
Python
12 votes226 views2 answers
How to read input from stdin in python
Python
10 votes233 views3 answers
Check python 32 or 64
Python
0 vote225 views1 answer
Flask mongoengine getting started
Python
0 vote215 views1 answer
Check if numpy array contains only duplicates
Python
2 votes248 views1 answer
How to align text in tkinter
Python
-2 votes228 views1 answer
How to comment out lines of code in python
Python
0 vote255 views1 answer
How to select li element in selenium python
Python
0 vote231 views1 answer
Python - matching people based on city
Python
0 vote248 views1 answer
Check if cuda is present in torch
Python