DVBsnoop EIT to XML to get EPG for channels

There are 39 replies in this Thread which was already clicked 3,404 times. The last Post () by pepsik.

    DVBsnoop powerful stream analysis tool.


    I need to extract EIT from the transport stream of a given transponder and then convert this data to xml format so that it can be included in the sources of the EPG Importer plugin.


    Is there anyone I could talk to and work out a solution together?dvbsnoop -n 1 -nph 0x0000 >/tmp/snoop_0x00.txt

    Code
    dvbsnoop -n 1 -nph 0x0000 >/tmp/snoop_0x00.txt
dvbsnoop -n 500 0x11 >/tmp/snoop_0x11.txt
dvbsnoop -n 500 0x12 >/tmp/snoop_0x12.txt


    I get to log the EPG from the TS stream on a given transponder.

    It only needs to get EPG/EIT for one channel and that for 12/24h

    OCTAGON SF8008, OATV 7.3, OPLI 9, Black Harmony

    Edited 2 times, last by azman26 ().

  • azman26

    Changed the title of the thread from “DVBsnoop EIT to XML to get epg for channels” to “DVBsnoop EIT to XML to get EPG for channels”.

    I need to extract EPG/EIT for channels from HB 13E transponder 12015 H using Octagon SF808 receiver.

    Channels VOD202 - VOD208.

    Currently, I manage to log everything that the transponder transmits.

    I'm only interested in EIT for VOD channels.

    I use filters and ae masks, I think I don't know enough about this tool (dvbsnoop) to use it effectively.


    My goal is to get the EPG for these channels and then add it to the sources of the EPG Importer plugin.

    I have been creating EPG for the Poland region for almost a year now and in cooperation with RytecTeam I am providing a guide for channels received in Poland.


    I miss the full happiness of mine and my EPG recipients of these VOD202-208 channels.


    So recently I came up with about DVBsnoop and EIT and I thought that with your help I could get a guide to the operator via sat and merge it with what I generate using WG++

    OCTAGON SF8008, OATV 7.3, OPLI 9, Black Harmony

    Edited 5 times, last by azman26 ().

    thumbs up 1
    • Official Post
    Quote from azman26

    Currently, I manage to log everything that the transponder transmits.

    Interesting, post your logged data.

    Is the issue logging the data you want or is it converting that data into an xml ?

    What STB or machine / tuner are you using dvbsnoop on ?



    I used to do EPG myself, but im going back many many years now, I moved away from Web Grab when they wanted money for new ini files., perhaps things have changed now...... anyways grabbing data from the satellite would be preferable as it is likely to be more accurate then a website.

    VOD202-208 channels are the only channels of the Polsat operator for which I cannot get EPG from other sources.

    I have a license for WG++ so I have access to ini files of various platforms from all over the world.


    I have DVBsnoop installed on the Octagon SF8008 receiver.


    Attached are the files from dvbsnoop logged 1000 lines from the transponder that broadcasts these channels.


    I only care about these channels.


    snoop_0x00.txt

    snoop_0x11.txt

    snoop_0x12.txt

    snoop_sec_nph_0x12.txt


    I create EPG for all channels dedicated to the region of Poland, including fixed channels, terrestrial TV, cable TV as well as for IPTV channels (TVP VOD, Rakuten TV and local niche regional IPTV channels)

    And just to be completely happy, I lack EPG for these few VOD channels of the Polsat Box platform.

    They are supposedly Video on Demand channels by name, but there is one movie broadcast 24/h for 7 days and then a change and so on each of these channels.


    I would appreciate any help in obtaining a guide from the EIT data. The next step will be to extract interesting data from these files and add them to EPG Poland sources. I think that together with my colleague doglover from RytecTeam, we will be able to enter the obtained data into the sources, and even if it were possible, I will get ratings from IMDB and from the Polish FilmWeb portal for events.


    -----------------------

    Some basic commands I use in dvbsnoop

    Other simple filter examples:

    dvbsnoop \-s sec \-f 0x4E \-m 0xFF 0x12


    Simple filter for some PID values, some examples:

    dvbsnoop \-n 5 \-nph 0x00 | grep \-i "PID: "

    dvbsnoop \-spiderpid \-nph \-n 10 0x00 | grep \-i "PID: " | sort | uniq

    dvbsnoop \-s signal \-pd 9 | ./perl-statistics-prog.pl


    Display EPG, 10 sections:

    dvbsnoop \-s sec \-nph \-n 10 0x12

    dvbsnoop \-s sec \-ph 3 \-n 10 -crc 0x12

    OCTAGON SF8008, OATV 7.3, OPLI 9, Black Harmony

    Edited 4 times, last by azman26 ().

    thumbs up 2
    • Official Post

    Correct me if i am wrong here, ( as said it has been a few years since i played about with any of this....)

    but for this to work correctly, then the data needs a start time and end time for each EPG event ?

    Some of the data is not right, there is special characters in it.



    I wonder if a reader could be added to the e2 radiotimesxmltvemulator plugin ?

    Perhaps abu baniaz could give us his thoughts here ??


    Here is the plugin in question

    RadiotimesXmltvEmulator/providers/skyuk_28.2.conf at gui-plugin · LraiZer/RadiotimesXmltvEmulator
    generate radtiotimes xmltv feed compatible data files directly from dvb stream data. - LraiZer/RadiotimesXmltvEmulator
    github.com
    Quote from jenseneverest

    text: "..magazyn<BR>Prowadz±: Rafa³ Dꢩ񳫩, Micha³ Wodzi񳫩<BR>Rafa³ Dꢩ񳫩 wraz z zaproszonym ekspertem przedstawia wydarzenia ka¿dej kolejki spotka񠰩³karskiej Ekstraklasy. Program rozpoczyna si꠴u¿ po zako񣺥niu ostatniego z sobotnich mecz󷮼BR>Od lat" -- Charset: ISO/IEC 8859 special table


    As for errors in the description of the event, these are Polish diacritics (ą, ć, ę, ł, ń, ó, ś, ż, ź).


    ------------

    Edit


    I'm looking at the Radiotimes Xml Emulator plugin you recommended.


    I need to decipher how to add more operators, e.g. "Cyfrowy Polsat"

    conf, dict and theme files.

    jenseneverest Do you know where I could read about it and learn a bit.


    If I managed to add the Cyfrowy Polsat operator, this plugin would do everything I want to achieve.

    OCTAGON SF8008, OATV 7.3, OPLI 9, Black Harmony

    Edited 3 times, last by azman26 ().

    thumbs up 1
    • Official Post

    The plugin is above my little brain, the .dict part looks complicated to say the least,

    the best thing would be to ask the plugin author on github if it could be done. I dont think he is on our forum ?

    I am not sure if openTV data is the same as EIT data??


    Alternatively you could clean the data you get via dvbsnoop with a regex script, again that is beyond me without a lot of googling.

    Perhaps Lululla or KiddaC would be able to help if they have some spare time.

    And again post after post, the admin of this forum probably doesn't like me anymore :)


    The topic moved away from the main question, which was how to obtain events for channels directly from the Transport Stream of a given transponder from the operator that transmits them in this transmission.


    There are channels for which it is not possible to obtain EPG from websites, but only from SAT or DVB-T broadcasts from the data that the operator transmits along with the video/audio stream.


    This data is in the EIT tables that are read by each Enigma2 receiver.

    In E2 we have something like epgcasche which reads this data and converts it to an EPG.DAT file


    I can read these EIT tables, but unfortunately only for now/next, and export them to a txt file.


    I started this thread so that maybe someone more familiar with the topic could tell me how to read more data from TS, at least for these 12 or 24 hours.


    With this data, I would be able to read the most important information about events and pass them through the IMDW and FW parsers, thus obtaining quite nice EPG for channels for which there is no guide on any website.



    --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------


    I think so, or we'll split this topic into two.

    One regarding the current EPG support using existing solutions.


    And the second one deals exclusively with the issue of how to obtain EPG from the operator's transmitted data.


    Currently, there is chaos in this topic and everyone writes how to obtain EPG from external sources.

    As I mentioned above, it is not difficult and complicated.


    Let's focus on how to obtain and convert EIT data transmitted by the operator into XML data.



    --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

    biko


    Zrzut ekranu 2023-09-08 225017.png


    EPG channels | WebGrab+Plus


    For which country and the operator providing satellite services there do you not have EPG.

    Please look at the list of websites with schedules and the countries for which they can be created in the WG+++ program.


    If you do not find what you are interested in on this list, please contact me and I will try to help you solve your problem.


    -----------------

    And finally, I apologize to everyone for my bad English.

    OCTAGON SF8008, OATV 7.3, OPLI 9, Black Harmony

    Edited 5 times, last by azman26 ().

    Thanks 1

    I think people know how webgrab works but people don't want to use it because of its closed source paywalled encrypted ini bs.

    The grabbers I posted are 100% opensource, community driven and already have huge support from the IPTV community.

    Us (the sat community) teaming up with them would be much more beneficial than running separate projects.

    It was just a suggestion.

    Sorry for going off topic, but as you can see, other agree.

    The topic concerns EIT included in TS sent by the operator via SAT.

    Please stick to the topic and do not argue about which is better or this or that.

    After reading for a while, you become familiar with the EPG topics and get what you want, whether from these sources or others.


    Gentlemen, we are discussing topics that are described on this or that forum and it is not a problem to obtain EPG from these or other sources.

    I repeat my request that we return to my main question about extracting EIT from the transmission stream and not deal with something that has been possible and easy to obtain for a long time.



    1_0_1_C25_1E78_71_820000_0_0_0_20230909000149.jpg1_0_1_C25_1E78_71_820000_0_0_0_20230909001427.jpg1_0_1_C25_1E78_71_820000_0_0_0_20230909001408.jpg1_0_1_C25_1E78_71_820000_0_0_0_20230909001350.jpg1_0_1_C25_1E78_71_820000_0_0_0_20230909001302.jpg1_0_1_C25_1E78_71_820000_0_0_0_20230909001211.jpg1_0_1_C25_1E78_71_820000_0_0_0_20230909001147.jpg1_0_1_C25_1E78_71_820000_0_0_0_20230909001108.jpg1_0_1_C25_1E78_71_820000_0_0_0_20230909001051.jpg1_0_1_C25_1E78_71_820000_0_0_0_20230909001026.jpg1_0_1_C25_1E78_71_820000_0_0_0_20230909000943.jpg



    Really, you can't find EPG for your channels in all these sources?

    That's millions of pieces of data for millions of channels.

    OCTAGON SF8008, OATV 7.3, OPLI 9, Black Harmony

    Edited once, last by azman26 ().

    thumbs up 1

    Let's just stick to the question and try to do something with it.


    Where do I get this knowledge about EPG and plugins related to EPG, if not from this or another forum with topics that describe exactly how these plugins work.


    -----------------------------


    I ask the moderator to ban all subsequent posts that are not related to the initial topic.

    OCTAGON SF8008, OATV 7.3, OPLI 9, Black Harmony

    Edited once, last by azman26 ().

    thumbs up 1

    On another forum, one of the users @CocoJambo found a project on GitHub of something the author called "epgsnoop".


    GitHub - hadleyrich/epgsnoop
    Contribute to hadleyrich/epgsnoop development by creating an account on GitHub.
    github.com


    After a preliminary review of the files, it seems that these scripts would be very useful and would do what I am thinking about, which is downloading data from EIT and saving it in an XML file. Moreover, it seems that it also downloads data from IMDB.


    Due to the fact that this project is already old, it should be refreshed

    - convert to py3

    - change the tuner "tuning" procedure.

    - and probably a few other errors that will appear during use.


    Could someone more familiar with PY and SQL take a look at this project and assess its possible use in E2?

    OCTAGON SF8008, OATV 7.3, OPLI 9, Black Harmony

    thumbs up 1
    • Official Post

    Interesting, that has been written exclusively for sky, in new Zealand, for use on a PC with tuner card.

    it would need some work but may be an option if our e2 tuners work correctly and for anyone with the time.

    My biggest worry is your only able to grab now and next data..... the dvb snoop command is about the same



    The attached windows programme would also work, either by a network stream, or again a PC with tuner card / usb tuner

    It looks like a great programme with many options, again not had time to test

    By modifying and improving epgsnoop, it would be possible to write conf files for each of the satellite operators.


    If using epgsnoop it was possible to create an xml file with the full schedule provided by a given operator for its platform, it could then be very easily merged in WG++ or EPG Importer.


    By obtaining a guide from EPGsnoop and from Rytec Team sources, we would get everything we want regarding EPG, even for channels for which no schedule can be found on websites.


    Then, using EPG Export (yes, version 1.5 works without any problem in PY3), the entire final EPG could be exported for e.g. KODI, VDR, Channels or other Android players.


    I'm just getting to know this epgsnoop but with the level of my knowledge it will probably take ages :)

    OCTAGON SF8008, OATV 7.3, OPLI 9, Black Harmony

    Edited 2 times, last by azman26 ().

    thumbs up 1
    • Official Post

    For epgsnoop to even warrant time spending on it, you need more than now and next epg data.

    Sky NZ, now has openTV epg from the radiotimesemulator plugin i linked to the other day.

    I just came across this project: https://github.com/mkrufky/libdvbtee

    It might be what you need?


    Not sure if its capable of getting more detailed info though... but does anyone actually read descriptions?

Participate now!

Don’t have an account yet? Register yourself now and be a part of our community!

Register Yourself Login