Question about the RSS spec:
When pulling RSS, how do you know how often to poll? How do you know which items have been seen previously?
When pulling RSS, how do you know how often to poll? How do you know which items have been seen previously?
https://www.rssboard.org/rss-specification#optionalChannelEl...
ttl, skipHours, skipDays
Source: working on a feed reader (https://lighthouseapp.io/) and feed finder tool (https://lighthouseapp.io/tools/feed-finder).
Tracking what's been read or not isn't done by the RSS feed or whoever hosts it, it's performed by the user's feed reader, which be just a local app on your phone or PC, or it might be a cloud service, either hosted (like Feedly) or self-hosted (using ie. FreshRSS).
I would guess a combination of frequency from sitemap.xml, last modified http header, and past heuristics. Previously viewed items would, I think, need to be cached in the client (unless the RSS URL uses some kind of token to identify the user, which sounds ripe for abuse).
Personally I just start my reader then it aggregates and sorts my feeds by date into a single interface. This works well specially for much larger numbers.
TL;DR: readers should not poll more often than once and hour, use ETag and If-Modified-Since to determine whether to download the full feed again.
Which items you have seen previously is something the feed reader keeps track of.
Is there a particular field that can be used as an identifier?
Of course a lot of feeds also get this wrong and change the GUIDs for existing entries once in a while which results in strictly compliant readers showing you the entire feed history as new. Really annoying.
The latter is annoying, I agree.
¹ It is an NNTP interface so the article is superseded; https://feedbase.org/about/ - if you don't want to see updates, you can configure your newsreader to skip supersedes.
It really depends on you but IMO for most feeds polling once a day is plenty.