Python web crawler

import scrapy

class BlogSpider(scrapy.Spider):
    name = 'blogspider'
    start_urls = ['https://blog.scrapinghub.com']

    def parse(self, response):
        for title in response.css('.post-header>h2'):
            yield {'title': title.css('a ::text').get()}

        for next_page in response.css('a.next-posts-link'):
            yield response.follow(next_page, self.parse)

Source: scrapy.org

Similar codes

0 vote372 views1 answer
Pandas convert text duration to minutes
Python
0 vote317 views1 answer
Python toupls
Python
9 votes315 views1 answer
Mac upgrade python to 3.8
Python
-1 vote314 views2 answers
Dataframe multiindex
Python
28 votes314 views3 answers
Pip install specific version
Python
80 votes325 views6 answers
Python code to exe file
Python
2 votes320 views1 answer
Filter by row contains pandas
Python
8 votes331 views2 answers
Tkinter how to make a root non rezizable
Python
1 vote347 views1 answer
Django login redirect
Python Django
0 vote335 views1 answer
Leetcode 206 python
Python