Dateless items readded after cleanup

[jtojnar]

In RSS feeds not containing dates the items were published, the items will be added again on the next update after they are finally collected as a garbage.

While this does not happen very often (four out of 150 feeds I subscribe to), it is still very annoying.

Example of such feed: http://www.webylon.info/feed

[niol]

This is strange because the test is done on the items uids and when it is lacking, the link is used as uid.

I cannot reproduce on my installation with the provided feed.

[jtojnar]

I do not mean that there are duplicates. The problem is that the items that were removed from database will get added with next update since selfoss has no way of telling that they were there previously.

You could reproduce it by adding the aforementioned feed, fetching the items, having the selfoss clean up (can be triggered, for example, by replacing sub for add here) and then fetching the source again.

One possible solution would be modifying the cleanup procedure to ignore the items that were just fetched.

[niol]

Thank you for the explanation, now I understand. The only solution I see would be to record the last date items were seen in a feed and cleaning up using this new date.

[jtojnar]

Yes, that is another solution. Now I realize my solution would not work when updating a single source – for the solutions to be equivalent updateSingle would have to limit the cleanup to the single source.

Your solution would require modifying the database, my would make the queries more complicated.

[aminecmi]

Not sure if its the same issue, but this feed produces the same issue.

The items are added multiple times to the database, and appear multiple times on the app.

[jtojnar]

@aminecmi That looks like a different issue, this one does not cause multiplication of items, only losing the information whether an item was read when it does not have a pubDate in the feed.

[aminecmi]

@jtojnar ok, I created a new issue #905 for this.