Python web crawler

import scrapy

class BlogSpider(scrapy.Spider):
    name = 'blogspider'
    start_urls = ['https://blog.scrapinghub.com']

    def parse(self, response):
        for title in response.css('.post-header>h2'):
            yield {'title': title.css('a ::text').get()}

        for next_page in response.css('a.next-posts-link'):
            yield response.follow(next_page, self.parse)

Source: scrapy.org

Similar codes

3 votes333 views3 answers
Convert list to string separated by comma python
Python
7 votes339 views4 answers
Python possible combinations
Python
4 votes353 views2 answers
How to concat on the basis of particular columns in pandas
Python
2 votes285 views2 answers
How to play audio in python
Python
10 votes366 views4 answers
File path current directory python
Python
4 votes341 views3 answers
Fibonacci sequence python code
Python
1 vote424 views1 answer
Django __str__ self multiple
Python Django
0 vote334 views1 answer
Python get current class name
Python
0 vote370 views1 answer
How to import packages in python
Python
0 vote321 views1 answer
Oauthlib python error
Python