2024-04-16 12:30 === :::info ***Next meeting: 2024-04-24 21:00 CET (9pm)*** ::: ## Subscriptions endpoint Difficult scenario, by Hans-Peter (ByteHamster): > What happens if client says 'new_feed_URL' is xyz, and the server already has another podcast in the database that also has URL xyz. This may happen if client A supports <iTunes new feed URL> syntax, and there is client B that doesnt. Client A submits to server new podcast subscription with the _new_ URL. Client B doesn't support it, so would create a subscription with the old URL. Now client B gets an software update, and then supports the redirect, and then tells the server there's new_feed_URL from old to new feed. Then the server has already two feeds and receives a feed that conflicts. > Similarly: some client follows temporary redirects while another client follows permanent redirect. * Subscription conflicts. Merge/deduplicate on client or on server side? * if `new_feed_url` is received, check if this already belongs to different podcast * server needs to have merge/migration process. (could add a description to the 'update subscription' endpoint: "if feed is already known in system") * should be a background job, as merging process takes a while * recommendation for server: * response format where client can request any changes since timestamp, where API replies with tombstone / resource has moved, and client updates the guid in their database * we agree to have timestamps at data point level, in order to support merging. * Specification already covers 'GUID update' (but that doesn't cover merge conflicts) * Currently each old URL inform is a single PATCH, so if there's a chain of redirects, that'd be a bunch of calls required. * We expect the clients to _always_ inform the server of past URLS `/subscriptions/{guid}/episodes` -> 301 HTTP resource moved permanently -> `/subscriptions/{new_guid}/episodes` -> clients SHOULD update guid ### Data timestamps :::info Following up on https://pad.funkwhale.audio/s/88C5eXrRq#General-considerations-to-data-points ::: * API should always return timestamp with every call, to support merging (discussed above) ```json { "total": 2, "page": 1, "per_page": 5, "subscriptions": [ { "feed_url": "https://example.com/rss1", "guid": "31740ac6-e39d-49cd-9179-634bcecf4143", "is_subscribed": true, "guid_changed": "2022-09-21T10:25:32.411Z", "new_guid": "8d1f8f09-4f50-4327-9a63-639bfb1cbd98" }, { "feed_url": "https://example.com/rss2", "guid": "968cb508-803c-493c-8ff2-9e397dadb83c", "is_subscribed": false, "subscription_changed": "2022-04-24T17:53:21.573Z", "deleted": "2022-04-24T17:53:21.573Z" } ] } ``` ```json { "total": 2, "page": 1, "per_page": 5, "subscriptions": [ "8d1f8f09-4f50-4327-9a63-639bfb1cbd98": { "feed_url": "https://example.com/rss1", "guid": { "old_value": "31740ac6-e39d-49cd-9179-634bcecf4143", "value": "8d1f8f09-4f50-4327-9a63-639bfb1cbd98", "changed_at": "2024-01-01T10:25:32.411Z" }, "is_subscribed": { "value": true, "changed_at": "20240101T10:25:32.411Z" } }, "968cb508-803c-493c-8ff2-9e397dadb83c": { "feed_url": "https://example.com/rss2", "guid": "968cb508-803c-493c-8ff2-9e397dadb83c", "is_subscribed": false, "subscription_changed": "2022-04-24T17:53:21.573Z", "deleted": "2022-04-24T17:53:21.573Z" } ] } ``` * `old_value`: only in case of feed URL and guid (at subscription & episode level) * `value` * `changed_at` ## Episode GUIDs Maybe not needed to always generate, as other fields will need to be sent anyway for identification/matching of new episodes with the RSS feed. At the same time, this matching is only needed for new episodes, subsequent data exchanges could rely on simple guid. For AntennaPod, the extra space needed is worth it helps to prevent many edge cases that we'd have without the guid. ## For next meeting ### Episode deduplication From notes of meeting with Hans-Peter (AntennaPod): > As AntennaPod, we'd be hesitant to accept incoming deduplication instructions. Because if an episode disappears in AntennaPod, we would get the comments from the user. Do we trust other clients? 'User listened to episode X' - that's so basic that we can more easily trust. Just accepting anything that the server does is a bit scary. A misbehaving client could render AntennaPod completely unusable, e.g. by merging all episodes into 1. > > What do we do in AntennaPod if there's a deduplication request coming in that we would ignore? Since the server & other clients already think the episode is gone, would we send a 'new episode' to the server? We could ask the user to go ahead with merge/deduplication or not. If rejected, we create a new episode. Then a "don't deduplicate flag" is applied to the episode. > > Alternatively, send deduplication as strong recommendation. If ignored, keep synchronising with the old episode - so server always has 2 episodes. Then in the UI of one of the clients there is one more > > Alternatively, we could keep duplicates on the server and work with 'tombstones' (rather: episode with deduplication flag with reference to the new one). Then sends back to the client: I think you want to use this other one instead, but if you insist you can use this deduplicated episode (but know that it won't propagate to other clients). Then two clients that agree while a third client has another opinion, the two can still use the 'tombstone'. And there's nothing to be deleted if it doesn't want to delete it.