This is an old revision of the document!
HOW TO REFRESH ONLINE FOLDERS ON DEMAND FROM THE CLIENT
THE NEED FOR REFRESH ON DEMAND
Serviio will automatically refresh the content online sources (feeds), with each restart and on a periodic basis according to the “Feed expiry interval (hours)” set in the console. This is adequate for Online Sources that are relatively static, but there are some sources (feeds)that are dynamic, with their feed items constantly changing, so the feed must be refreshed in order for Serviio to obtain and display the latest set of feed items. Sports Event sites such as HaHaSport and News sites such as Video.AP are two examples of this.
There are two ways to initiate those more frequent refreshes.
One approach is to create a WebResource groovy for the feed and set the expiry date to initiate refreshes of that feed on periodic basis consistent with the known dynamics of the source for example just before the start of the next scheduled sports event or every 30 minutes for a news feed.
The Online Source HAHASPORT.COM is an example. This site maintains and provides a list of scheduled events by sport, and provides links to embedded player streams of events as they go live. To maintain currency of a Serviio menu, the list of scheduled events must be reviewed at least hourly since new events can be added or removed at any time; live events must be detected minutes prior to their scheduled start, and if not yet started, again at their scheduled start and a few minutes thereafter; and then live events must be removed a few minutes after their scheduled end time.
The problem with this approach is the continual use of resources to refresh and maintain the current set of feed items even though they may never be used. Secondly, this constant stream of refreshes particularly when multiple sport feeds are enabled, is beyond the capacity of most systems and feed refreshes are constantly backlogged due to insufficient time to access the changing feed items.
The better approach is to be able to “Force Refresh” a feed just prior to its use. However the problem with this approach is that if a significant number of new feed items have been created since the previous refresh, it may take Serviio many minutes to refresh and make the feed available again. Another problem is that while the Serviio Server Console and various Remote Device Console implementations offer the ability to “Force Refresh” Online Sources On Demand, no capability is offered to initiate a “Force Refresh” on demand from the menus on a Client device.
The ultimate solution therefore is to implement those dynamic feeds with a Web Resource that refreshes on a periodic basis that will minimize the number of new feed items to be added by a “Force Refresh”, and to provide a “Force Refresh” capability from the menus on the client device, that may be used just prior to using the feed.
The facility to initiate periodic refreshes more frequently than the console default refresh value is documented in the “Serviio Online Resource plugin implementation Guide” under “Date expiresOn”. Date is based on the current time {long curTimea = System.currentTimeMillis()} plus the milliseconds to the desired expiry time. Note that expiry will be on the earliest of that Date and the console default, and the refresh will actually be performed by Serviio 5 minutes ahead of that time.
The facility to initiate “Force Refresh” from the menus on the client device is documented below, and consists of a WebResource RefreshWebResources.groovy that generates a menu of “Force Refresh” feed items for all WebResource feeds in the Serviio library, which allows one or more feeds to be refreshed on demand, and a “forcerefresh” method that may be included in any WebResource groovy to create a “Force Refresh” feed item within the Webresource menu itself.
Consistent with this approach the HAHASPORT.GROOVY implements an automatic expires on refresh every 60 minutes plus the method which creates a “Force Refresh” feed item and may be reviewed as an example of the methodology.
ON DEMAND METHODOLOGY
The ability to implement “Force Refresh” feed items in the RefreshWebResources.groovy or the “forcerefresh” method in any WebResource is based on 4 prerequisite implementations:
1) The installation of CURL.EXE which provides the abilty to
The prerequisite is the development of a WebResource groovy for the online source and the identification of a permanently available online source link that may be used to provide a Refresh Menu Item stream.
The WebResource groovy should create the Refresh item as the first menu item, using an appropriate thumbnail and title and the Refresh Menu item stream.
If it is desired to automatically refresh the folder more frequently than the default expiry time in the console, the WebResource groovy should also generate a value for “expires on” for at least one of the items equal to that value + 5 minutes else Serviio will automatically refresh using the console value. (Note that Serviio will not allow expiry to be deferred beyond the console period.)
The ServiioService.exe.vmoptions file also needs to be edited to cause ffmpeg.exe transcoding requests to be sent to a script which first checks for the link to the Refresh Menu item stream and initiates an Online Library Refresh of the appropriate library id using CURL before calling ffmpeg.exe to transcode the requested stream.
Finally the script needs to be edited to define the library id and the corresponding Refresh Menu Item stream url, so that the correct library is refreshed when the Refresh Menu item is clicked.
THE HAHASPORT IMPLEMENTATION
The HAHASPORT.GROOVY uses the c- value designating the sport from the ResourceUrl eg:http://www.hahasport.com/c-1.html to request different Refresh Menu item streams for each sport. Similarly the script checks for each of these streams to determine the online library id to refresh.
The HAHASPORT.GROOVY uses “https://sites.google.com/site/serviiorss/refresh.jpg” as the thumbnail, “Click to Refresh” as the videotitle and “rtsp://a1709.l1856953708.c18569.g.lm.akamaistream.net:554/D/1709/18569/vn/reflector:53708” (which is the view from the ISS) as the Refresh Menu item stream where the n in vn corresponds to the c- value in the ResourceUrl.
With these changes clicking on the Refresh menu item, initiates a refresh of the corresponding online source while the Refresh Menu Item stream plays. If one waits for the refresh to complete before stopping playback, the refreshed menu will be displayed on return and the desired item selected for playback, else the menu will be unavailable on return and Serviio will exit and require a restart to access the updated menu once available. (Note that a refresh can require up to 30 seconds to cache a new item, but with hourly automatic updates many of the events should already be cached).
The HAHASPORT.GROOVY is dependent upon the attached ffmpeg.bat script with the appropriate online library id's added as determined by using CURL to dump the repository as documented in the script, and the script is similarly dependent upon the change to ServiioService.exe.vmoptions as documented in the script. (Note I am sure more advanced scripts can be created by those conversant with other scripting tools than bats)
After listing the available live streams, the HAHASPORT.GROOVY also generates a final item in each menu, showing either the scheduled start time of the next event or the absence of any more events in the schedule for that sport, along with the time of the next automatic refresh.
USING THE HAHASPORT GROOVY
To use the HAHASPORT.GROOVY hahasport.groovy.v2.zip just add the following Resource URL's as WebResources in the Serviio Console, together with the appropriate labels.
http://www.hahasport.com/c-1.html FOOTBALL
http://www.hahasport.com/c-3.html HOCKEY
http://www.hahasport.com/c-4.html TENNIS
http://www.hahasport.com/c-6.html BASKETBALL
http://www.hahasport.com/c-7.html BASEBALL
http://www.hahasport.com/c-8.html GOLF
http://www.hahasport.com/c-9.html CYCLING
http://www.hahasport.com/c-10.html MOTORSPORTS
http://www.hahasport.com/c-11.html OTHER
Since each live event may have multiple streams available, each from a different source, an estreams parameter may be used with the ResourceUrl to cause multiple streams to be displayed for each live event if available. estreams may equal 1 thru 9 with 9 signifing that all available streams should be displayed, for example http://www.hahasport.com/c-1.html?estreams=9
Note also that since HAHASPORT.COM sources its streams from many different sources, there may be live streams that the groovy does not decode for display in the Serviio menu. For details on the stream decoding set the onlinelibrary log to debug mode.
To access the current events for a given sport, open the menu and click on the Refresh menu item which will initiate a refresh of the online source while the Refresh Menu Item stream plays. Wait for the refresh to execute (30 seconds per new item) before stopping playback and the refreshed menu will be displayed on return and the desired item selected for playback. If playback of the Refresh Menu Item stream is stopped before the refresh is complete the menu will be unavailable on return and Serviio will exit and require a restart to access the updated menu once available.
Note that if you do not to implement the changes required to support the Refresh On Demand menu item, you must use the Serviio console or a remote console to force the refresh of each sport in order to obtain the current live stream event menu.
THE SCRIPT
@echo off : : To intercept Serviio FFmpeg Calls install this bat as "C:\Program Files\Serviio\lib\ffmpeg.bat" : then Edit "C:\Program Files\Serviio\bin\ServiioService.exe.vmoptions" : and change the Dffmpeg.location variable to : -Xmx384m : -Dffmpeg.location="C:\Program Files\Serviio\lib\ffmpeg.bat" : : Finally install Curl as required in C:\windows\system32 and use : Curl -X GET "http://localhost:23423/rest/repository" > C:\temp\repository.txt : to get the online repository ids for each c- value in http://www.hahasport.com/c-n.html : and enter the ids in the table below. eg: set C1=54 set C1= set C2= set C3= set C4= set C5= set C6= set C7= set C8= set C9= set C10= set C11= : The following code will then be executed before passing control back to Serviio's ffmpeg. set A=%1 shift set B=%1 shift set C=%1 shift set D=%1 shift set E=%1 shift set F=%1 shift set G=%1 shift set H=%1 shift set I=%1 shift set J=%1 shift set K=%1 shift set L=%1 shift set M=%1 shift set N=%1 shift set O=%1 shift set P=%1 shift set Q=%1 shift set R=%1 shift set S=%1 shift set T=%1 shift set U=%1 shift set V=%1 shift set W=%1 shift set X=%1 shift set Y=%1 shift set Z=%1 shift set AA=%1 shift set BB=%1 shift set CC=%1 shift set DD=%1 shift set EE=%1 shift set FF=%1 shift set GG=%1 shift set HH=%1 shift set II=%1 shift set JJ=%1 shift set KK=%1 shift set LL=%1 shift set MM=%1 shift set NN=%1 shift set OO=%1 shift set PP=%1 shift set QQ=%1 shift set RR=%1 shift set SS=%1 shift set TT=%1 shift set UU=%1 shift set VV=%1 shift set WW=%1 shift set XXY=%1 set CODE=NONE echo %A% %B% %C% %D% %E% %F% %G% %H% %I% %J% %K% %L% %M% %N% %O% %P% %Q% %R% %S% %T% %U% %V% %W% %X% %Y% %Z% %AA% %BB% %CC% %DD% %EE% %FF% %GG% %HH% %II% %JJ% %KK% %LL% %MM% %NN% %OO% %PP% %QQ% %RR% %SS% %TT% %UU% %VV% %WW% %XX% %YY% %ZZ% >> "C:\Program Files\Serviio\log\ffmpeg.log" :COM Note Serviio appends ?tcp to rtsp links. if .%B% == .rtsp://a1709.l1856953708.c18569.g.lm.akamaistream.net:554/D/1709/18569/v1/reflector:53708?tcp set CODE=%C1% if .%B% == .rtsp://a1709.l1856953708.c18569.g.lm.akamaistream.net:554/D/1709/18569/v2/reflector:53708?tcp set CODE=%C2% if .%B% == .rtsp://a1709.l1856953708.c18569.g.lm.akamaistream.net:554/D/1709/18569/v3/reflector:53708?tcp set CODE=%C3% if .%B% == .rtsp://a1709.l1856953708.c18569.g.lm.akamaistream.net:554/D/1709/18569/v4/reflector:53708?tcp set CODE=%C4% if .%B% == .rtsp://a1709.l1856953708.c18569.g.lm.akamaistream.net:554/D/1709/18569/v6/reflector:53708?tcp set CODE=%C6% if .%B% == .rtsp://a1709.l1856953708.c18569.g.lm.akamaistream.net:554/D/1709/18569/v7/reflector:53708?tcp set CODE=%C7% if .%B% == .rtsp://a1709.l1856953708.c18569.g.lm.akamaistream.net:554/D/1709/18569/v8/reflector:53708?tcp set CODE=%C8% if .%B% == .rtsp://a1709.l1856953708.c18569.g.lm.akamaistream.net:554/D/1709/18569/v9/reflector:53708?tcp set CODE=%C9% if .%B% == .rtsp://a1709.l1856953708.c18569.g.lm.akamaistream.net:554/D/1709/18569/v10/reflector:53708?tcp set CODE=%C10% if .%B% == .rtsp://a1709.l1856953708.c18569.g.lm.akamaistream.net:554/D/1709/18569/v11/reflector:53708?tcp set CODE=%C11% ::if .%CODE%==.NONE "C:\Program Files\Serviio\lib\ffmpeg.exe" %A% %B% %C% %D% %E% %F% %G% %H% %I% %J% %K% %L% %M% %N% %O% %P% %Q% %R% %S% %T% %U% %V% %W% %X% %Y% %Z% %AA% %BB% %CC% %DD% %EE% %FF% %GG% %HH% %II% %JJ% %KK% %LL% %MM% %NN% %OO% %PP% %QQ% %RR% %SS% %TT% %UU% %VV% %WW% %XX% %YY% %ZZ% if NOT .%CODE%==.NONE echo REFRESH %CODE% >> "C:\Program Files\Serviio\log\ffmpeg.log" if NOT .%CODE%==.NONE curl -X POST "http://localhost:23423/rest/action" -H "Content-Type: text/xml" -d "<action><name>forceOnlineResourceRefresh</name><parameter>%CODE%</parameter></action>" "C:\Program Files\Serviio\lib\ffmpeg.exe" %A% %B% %C% %D% %E% %F% %G% %H% %I% %J% %K% %L% %M% %N% %O% %P% %Q% %R% %S% %T% %U% %V% %W% %X% %Y% %Z% %AA% %BB% %CC% %DD% %EE% %FF% %GG% %HH% %II% %JJ% %KK% %LL% %MM% %NN% %OO% %PP% %QQ% %RR% %SS% %TT% %UU% %VV% %WW% %XX% %YY% %ZZ% exit
