DVBsnoop EIT to XML to get EPG for channels

There are 39 replies in this Thread which was already clicked 3,302 times. The last Post () by pepsik.

  • I have this plugin installed and I tried to do something with it, unfortunately with no results.


    Log from Radio Times Emulator


    polsatbox_13.0.log


    Does anyone know how to check in what standard a given operator broadcasts its EPG data?

    From the information I have:

    13E, Cyfrowy Polsat S.A. (Polsat Box) - OpenTV

    13E, NC+ (now Warner Bros. Discovery) - MHW

    It would be best if I could somehow verify this because I have this information from several years ago.

    OCTAGON SF8008, OATV 7.3, OPLI 9, Black Harmony

  • Thanks Chris230291


    Now I have to focus on this libdvbtee and extract what interests me most from the data.


    I am personally only interested in channels for Polish recipients.

    But nothing stops other interested parties from also starting to work in this direction and doing something for their own needs.


    As a result, maybe we will be able to create a nice plugin for obtaining EPG from EIT tables.


    Due to my age and the slower acquisition of new knowledge, someone of you will probably understand it much faster than me :)


    --------------------------------

    Edit


    It just takes me much longer to understand and put it together than it does for you young people.

    Don't get me wrong, I want someone to write a plugin right away. Let everyone do their own thing. The first one to publish an interesting solution should be involved.

    OCTAGON SF8008, OATV 7.3, OPLI 9, Black Harmony

    Edited once, last by azman26 ().

    • Official Post

    Doesn't e2 already scrape data over the air?

    I don't know because I don't use e2.

    I am just piping in a raw transponder stream from TVHeadend (using curl) into that program to get that data.

    Yes, simply tuning into the channels that azman26 is interested in, will get data for some of them.....

    See here:

    enigma2/lib/dvb/eit.cpp at master · openatv/enigma2
    openatv-gui. Contribute to openatv/enigma2 development by creating an account on GitHub.
    github.com

    The biggest issue with e2, is we do not had the full transport stream available, so things like dvbsnoop, do not work as it should.

    Would be interesting to see what libdvbtee comes up with as a network stream.

  • I know about it eit.cpp and epgcashe.cpp.


    And of course, EPG via Sat from the Enigma2 operator processes and displays it error-free.


    However, if we turn off the acquisition of data from EIT and MHW in the system settings and decide only on EPG from external sources provided to us by the RytecTeam using the EPG Importer plugin, so that the data from external sources richer in information is not overwritten with those poorer from the TS operator via sat, this mechanism does not work.


    1_0_1_3DCD_640_13E_820000_0_0_0_20230910235424.jpg


    My idea is to use EPG data from external sources, enter all the information in EPG.dat that E2 uses, and add what is not available on websites with TV schedules.


    There are several, a dozen or so channels, maybe several dozen throughout the entire orbit, for which we are unable to find schedules on websites.


    To show what it looks like, I will post a comparison of what the operator broadcasts with what we have from external EPG sources.


    Operator EPG

    1_0_1_290E_1EDC_71_820000_0_0_0_20230910235910.jpg



    EPG from an external source with the addition of ratings from IMDB and FW (Polish website about films and series)


    1_0_1_290E_1EDC_71_820000_0_0_0_20230910235801.jpg


    Sorry for the screenshots but I can only make them for Polish channels.


    When generating EPG for channels, I use the following data manipulation options, I can freely rearrange them to obtain the most optimal final result that satisfies everyone.

    Code
    <title>'title'{ - 'subtitle(, )'}</title>
    <desc>{[ 'category(, )' ] }{('episode(, )') }{'country(, )' }{'productiondate(, )' }{('rating(, )')}\n{FilmWeb: 'mdb-starrating(, )'  }{IMDB: 'starrating(, )'}{\n\n'description'}{\n\nReżyseria: 'director(, )'}{\nWystępują: 'actor(, )[max_elements=5]'}</desc>


    Also, when generating EPG for IPTV channels and regional niche channels or those from platforms such as FAST and IP Radio, there is no problem as long as this data is presented on the TV program pages.


    The problem is that if the operator broadcasts EPG data in EIT tables in the TS stream and there are no websites or other possibilities of obtaining EPG for such channels, the only thing left to do is log the operator's EIT data and then process them appropriately and make them public together with RytecTeam sources for a wider audience.


    And that's what I'm still wondering about: how to obtain EPG data from the EIT tables for these several/dozen channels, because the operator sends them.


    Obtain this data, process it appropriately and add it to RytecTeam sources or make it public for other recipients.

    OCTAGON SF8008, OATV 7.3, OPLI 9, Black Harmony

    Edited once, last by azman26 ().

  • Is the URL you guys use for iCAM stream relay not the full ts?

    • Official Post

    We get full audio, video ETC....But there is a load more in the full stream

    See this old thread about it

    Record complete transport stream. - [EN] Enduser support
    Page 1 of 5 - Record complete transport stream. - posted in [EN] Enduser support: In OpenPLi s there any way I can record the complete transport straight to…
    forums.openpli.org

    From what i have read, the STB chip makers are not interested in adding support for the full stream.....

    Funny you mention icam, i have always said the patch for that issue should be in the DVB drivers, not oscam.......

  • If the full stream, or at least all of the audio, video and encryption stuff is included for stream relay to work, are you sure the rest of the data is removed?

    Try running dvbtee like I was, but use one of the links you guys have in your special bouquets for iCAM.


    Code
    curl <iCAM stream relay url> | dvbtee/dvbtee -e -t10 -j


    Also, if you can enable a sat>IP server in e2, then it should be everything. I don't see how a client could work properly if it stripped everything out.

    I wonder why you would need to patch drivers for iCAM on e2?

    On PC we just patch libraries (libdvbcsa ) and software to support the non dvb standard.

    No drivers are touched.

    Doesn't e2 use the same libraries, or an equivalent?

  • The problem occurs when we have an E2 receiver and in other rooms of our house there are other receivers not from E2, but some Smart TV, MI TV BOX or NVIDIA SHIELD or some Home Theater on various systems based on Smart TV.


    Samart TV, Plex Server, KODI on OpenELECT and other multimedia solutions. It's easy to add an audio/video stream to anything.


    However, obtaining EPG is not that simple.


    Here's an example: Stalker Portal on Android players works with EPG, while in E2 we have video, but not necessarily EPG and in most cases it cannot be obtained. Why ? even though it is broadcast. And because it has a completely different data format, it is usually a channel ID than in the case of E2.

    OCTAGON SF8008, OATV 7.3, OPLI 9, Black Harmony

  • So that we also know what this XML file looks like for E2 and for other players, whether on Android TV or smart TV.


    For all smart receivers except E2


    pl.xml


    For any image on Enigma2 (some old file because my terminal is currently raking events from websites with guides)


    azman_basic.xml


    And who will discover what the difference is in these files?


    First of all, for E2 to correctly interpret this data, an additional file is required explaining that the XMLID data belongs to a channel with references of this type and not others.


    In the case of all other players and players not based on E2, the channel ID and its various other names included under the channels id tag are sufficient.

    Code
      <channel id="TVP 1">
        <display-name lang="pl">TVP 1</display-name>
        <display-name lang="pl">TVP1</display-name>
        <display-name lang="pl">PL: TVP 1</display-name>
        <display-name lang="pl">PL: TVP 1 HD</display-name>
        <display-name lang="pl">TVP 1 HD</display-name>
        <display-name lang="pl">TVP 1 FHD</display-name>

    OCTAGON SF8008, OATV 7.3, OPLI 9, Black Harmony

  • Quote

    Stalker uses a json formatted epg file.

    In most cases, yes, but they also broadcast better or worse EPG in the xmltv format.


    Which is not supported by E2 due to a different approach to channels_id and there is no translation into references, and E2 relies on channel references and adds EPG data to them.


    --------------------

    P.S


    Gentlemen, thanks for the creative discussion, I hope that some people will learn how EPG works in E2.


    But back to the main story


    libdvbtee is written in C++


    Either I'm wrong and I don't have much knowledge, but it can't be ported to E2 that easily.


    What remains is good will and the willingness to create EPG on an external machine by people who undertake it using this tool.


    Please correct me if I'm wrong.


    Do you think it would be better to develop EPGsnoop or Radio Times Emulator?


    Can I use libdvbtee on an external device and publish the resulting data somewhere on GH or another WWW share?

    OCTAGON SF8008, OATV 7.3, OPLI 9, Black Harmony

  • You can try to install dvbtee over ssh using the instructions posted on the GitHub.

    Or you can use an external device if you know of a way to stream from e2 to external devices.

    dvbtee also accepts files so you can use recordings too.

  • Where do I get this knowledge about EPG and plugins related to EPG,

    Who or what is stopping you from looking at the source code of any image and using the functionality of the epgcache.cpp module write the plugin you need for exporting EPG data from eEPGCache to XMLTV... If you write a console script, it will take at most 100 lines of Python code... If you add “menus”, then 150 lines =)


    enigma2/lib/dvb/epgcache.cpp at Python3.11 · BlackHole/enigma2
    openbh-gui. Contribute to BlackHole/enigma2 development by creating an account on GitHub.
    github.com

    XMLTVFormat - XMLTV


    This is 2-3 hours of work, provided that you are fluent in Python and are somehow able to read code in C


    p.s. If you, for example, are a “pervert” like me, then you can write a converter of “binary” data from the epg.dat file to XMLTV using the

    python struct module ... The structure of the epg.dat file can be obtained from the same epgcache.cpp code or analysis, for example, of this code - https://github.com/OpenPLi/eni…r/src/EPGImport/epgdat.py The only thing you should consider when analyzing this code is that all new images use the EpgDatV8 format but these are not very big differences, there are a couple of bytes to “push apart”

Participate now!

Don’t have an account yet? Register yourself now and be a part of our community!