Python web crawler

import scrapy

class BlogSpider(scrapy.Spider):
    name = 'blogspider'
    start_urls = ['https://blog.scrapinghub.com']

    def parse(self, response):
        for title in response.css('.post-header>h2'):
            yield {'title': title.css('a ::text').get()}

        for next_page in response.css('a.next-posts-link'):
            yield response.follow(next_page, self.parse)

Source: scrapy.org

Similar codes

10 votes263 views4 answers
Opencv resize image
Python
3 votes258 views1 answer
How ot split a string every fourth eter
Python
0 vote242 views1 answer
TypeError at /admin/auth/user/
Python
0 vote247 views1 answer
Sorted key len python
Python
0 vote212 views1 answer
Pyqt5 qpush button pressed
Python
2 votes292 views1 answer
Django filter multiple fields
Python Django
19 votes251 views3 answers
Convert a list ['Pythons syntax is easy to learn', 'Pythons syntax is very clear'] to a string using ‘&’.
Python
9 votes267 views6 answers
Convert dataframe to float
Python
1 vote237 views2 answers
Download pdf from url python
Python
1 vote244 views1 answer
Python name 'List' is not defined
Python