In order to play on-line resources via DLNA (those feeds declared as On-line Streams, RSS-Atoms, and Web Resources in the console) Serviio must determine the attributes of and optionally, the thumbnail for, each individual feed item (stream) in order for it to be displayed in the DLNA menu on client devices.
Hence, when a feed is added or enabled in the console, Serviio must first build a list of the feed item (stream) url's. For on-line streams this is the single url declared in the console. For RSS-atoms and Web Resources, Serviio first matches the declared feed url against the users installed plug-ins. If an RSS-atom feed url matches a plug-in then the Serviio accesses the feed url to obtain the map of links that must be provided by the feed url, and Serviio then executes the plug-in ContentURLContainer method to extract the feed item (stream) url's from those links. If an RSS-atom does not match a plug-in, Serviio accesses the feed url and extracts the list of feed item (stream) url's contained in the RSS feed url. If a Web Resource feed url matches a plug-in, the plug-in WebResourceContainer method is executed to first build a map of feed item links and then the plugin ContentURLContainer method is executed to extract the feed item (stream) url's from those links.
Once Serviio has obtained a list of the feed item (stream) url's, it then checks to see if the attributes of those url's have been previously stored in its cache (database). If not, ffmpeg is executed to obtain the feed item attributes and store them in the cache.
Since the feed items (streams) provided from many sources frequently change, Serviio establishes an expiry date for each feed, being the lesser of the console expiry interval or the feed items expiry defined by the plug-in. When that expiry interval is reached, Serviio will refresh the feed, by repeating the above sequence. Refreshes can also be individually forced in the console for any enabled feed and each enabled feed will be automatically refreshed when Serviio starts.
Once a feed has been processed, its feed items extracted, and their attributes cached, Serviio has the information required to provide any client device with the names of the available feeds, the list of the feed items, and based on the client profile, the format that will be sent to the client for each item. The client user may then select a feed item and the client will request that feed item from Serviio if it is in an acceptable format for the client. Serviio will then send the item in native format or transcode it in real time to the format determined by the client profile. If Serviio is unable to access the online feed item and the item has been defined as “expires immediately” by a plugin, Serviio will again attempt to extract the feed item (stream) url using the plugin ContentURLContainer method to obtain the latest valid url in order to access and send the item to the client.
This is an excellent design for managing online resources. With the immediate availability of the feed item url for on-line streams, and with the availability of all links with the single access of the rss url for valid rss links, the building and refresh of feed item url's is near instantaneous.
Note however that feeds utilizing plugins require considerably longer, because the ContentURLContainer method extract typically requires multiple url accesses and significant processing to determine each feed item url, and the current refresh logic not only extracts any new feed item urls, but also repeats the multisecond re-extract of formerly extracted items to re-obtain their feed item urls. This can require many seconds for each feed, and with the periodic refresh of many enabled feeds will utilize considerable resources and delay the availability of feed items during these extended duration refreshes.
For feeds with relatively static items, where refreshes need only be done occasionally, this is not a problem. A problem arises however for those feeds, like news and sports feeds, that are constantly adding new items, and must be refreshed frequently in order for those new items to be recognized by Serviio, because of the load imposed by the frequent re-extraction of urls for existing items.
Fortunately, Serviio allows feed item urls to be declared as “expires immediately” which results in the automatic re-extraction of the feed item url prior to any playback request. This allows a process to be implemented within plugins to declare feed items as “expires immediately” and to cache any extracted urls's for reuse on subsequent refreshes, thus avoiding the time required to re-extract existing urls. This results in feed refreshes that take one or two seconds rather that the multiple minutes required for re-extractions. That “Quick Refresh” process is described in the following wiki entry http://wiki.serviio.org/doku.php?id=quick_refresh.