undefined | Better HN

0 pointsjstanley4mo ago0 comments

I tried to make a custom Subscriptions view for YouTube but they kept blocking me for trying to scrape hundreds of RSS feeds.

0 comments

pbalau4mo ago

Maybe because their robots.txt disallows that?

    User-agent: *
    ...
    Disallow: /feeds/videos.xml
    ...

j / k navigate · click thread line to collapse

0 pointsjstanley4mo ago0 comments

I tried to make a custom Subscriptions view for YouTube but they kept blocking me for trying to scrape hundreds of RSS feeds.

pbalau4mo ago

Maybe because their robots.txt disallows that?

    User-agent: *
    ...
    Disallow: /feeds/videos.xml
    ...

j / k navigate · click thread line to collapse