Python web crawler

import scrapy

class BlogSpider(scrapy.Spider):
    name = 'blogspider'
    start_urls = ['https://blog.scrapinghub.com']

    def parse(self, response):
        for title in response.css('.post-header>h2'):
            yield {'title': title.css('a ::text').get()}

        for next_page in response.css('a.next-posts-link'):
            yield response.follow(next_page, self.parse)

Source: scrapy.org

Similar codes

0 vote329 views1 answer
Hex string to hex number
Python
0 vote372 views1 answer
Flask migrate multiple heads
Python Flask
0 vote316 views1 answer
Tf.data.Dataset select files with labels filter
Python
0 vote347 views1 answer
Streamlit headings;streamlit text
Python
0 vote319 views1 answer
Load png to python
Python
0 vote314 views2 answers
How to download file in python
Python
0 vote345 views2 answers
User logout in django rest framework
Python Django
0 vote355 views1 answer
Python legend outside
Python
0 vote318 views1 answer
Cast image to float32
Python
0 vote311 views1 answer
Combine column in csv python pandas
Python