E2m3u2bouquet - discussion & support

enigma2-plugin-extensions-e2m3u2bouquet_1.7.7_all.ipk

Works perfectly definitely epg downloads very fast openatv 7.4 vu+soloSE v2

Cloud-ibox Vu+Solo clone 333mhz Openpli4 it was necessary to completely clean the old files and after a restart it came out in the plugins

Everything works normally and the translation is also fine. It can't happen quickly with epg here normally 2 minutes :D.

Hello,

I need help to understand why a part of an epg isn't loaded by epgimport. The events are loaded but the last event loaded for this case is:

  <programme channel="5812bd9f249444e05d09cc4e" start="20240701075700 +0000" stop="20240701085900 +0000">
    <title>Cabin on the Sea: Newfoundland</title>
    <live/>
    <episode-num system="original-air-date">20240701075700 +0000</episode-num>
    <desc>Calm waters surround a tiny, secluded, red cabin on the sea.</desc>
    <icon src="https://images.pluto.tv/series/612d32391685a800136f0cd5/tile.jpg?fill=blur&amp;fit=fill&amp;fm=jpg&amp;h=900&amp;q=75&amp;w=900"/>
    <date>20201005</date>
    <series-id system="pluto">612d32391685a800136f0cd5</series-id>
    <category>Mindfulness &amp; Prayer</category>
    <category>Series</category>
    <category>Faith and Spirituality</category>
  </programme>

The next event and so every event after it are not loaded:

  <programme channel="5812bd9f249444e05d09cc4e" start="20240701085900 +0000" stop="20240701095900 +0000">
    <title>Change Islands Iceberg: Newfoundland</title>
    <live/>
    <episode-num system="original-air-date">20240701085900 +0000</episode-num>
    <desc>Footage of a lone iceberg in choppy waters.</desc>
    <icon src="https://images.pluto.tv/series/612d31c51685a800136f03a8/tile.jpg?fill=blur&amp;fit=fill&amp;fm=jpg&amp;h=900&amp;q=75&amp;w=900"/>
    <date>20200701</date>
    <series-id system="pluto">612d31c51685a800136f03a8</series-id>
    <sub-title>Lone iceberg, Change Island, Newfoundland</sub-title>
    <category>Mindfulness &amp; Prayer</category>
    <category>Series</category>
    <category>Faith and Spirituality</category>
  </programme>

I can't see and found a problem of syntax. I've attached the extract of the xml epg file.

Thanks for your help.

Quote from jeepcook

<live/>

??

Quote from pepsik

Quote from jeepcook

<live/>

??

Yes I saw that, but each event for this channel has this tag and some are loaded in the epg. I concluded the problem is elsewhere.

I attached the full Pluto TV US xml, and there is 3109 times this tag.

Quote from jeepcook

Yes I saw that, but each event for this channel has this tag

It's not a "tag" (the correct name is "node" or "element")- it's an error in the structure of the XML file...A Node is a part of the DOM tree, an Element is a particular type of Node e.g. <foo> This is Text </foo> ... In your example there is a “something” that has nothing to do with either “node” or “element”, this “something” is a clear error in the syntax and semantics of the XML file you provided ... I don't see any other errors in the "programme" element of the XML file you provided ... It all depends on which parser is used lxml or сElementTree (ElementTree) ... lxml “knows how” to ignore such errors ...

Yes it's a node sorry.

Currently I'm doing some tests after removing this <live/>

I will do a feedback.

Quote from jeepcook

Currently I'm doing some tests after removing this <live/>

I will do a feedback.

If this is used with e2m3u2b then simply install LXML... The e2m3u2b code will automatically recognize the presence of this library and will use it by default for XMLTV parsing ... LXML - can ignore errors in XML structure

opkg install python3-lxml

Restart GUI

After tests I think I have few problems with the content of my epg xml.

I have this in the e2m3u2b log:

[01.07 11:24:01] [e2m3u2b] [Favoris]: Channel table for 223 id has been formed
[01.07 11:24:01] [e2m3u2b] [Favoris]: Fetching EPG data: http://localhost/static/epg/epg.xml 
[01.07 11:24:01] [e2m3u2b] 89MB free RAM detected
[01.07 11:24:05] [e2m3u2b] XML data type detected
[01.07 11:24:05] [e2m3u2b] [Favoris]: Import EPG events started using threads, yay! 🚀
[01.07 11:24:05] [e2m3u2b] [Favoris]: Keep outdated EPG events to -1 day, 0:00:00
[01.07 11:24:05] [e2m3u2b] [Favoris]: Processing events information ...
[01.07 11:24:15] [e2m3u2b] Processed: 20615 events
[01.07 11:24:25] [e2m3u2b] Processed: 44363 events
[01.07 11:24:35] [e2m3u2b] Processed: 80955 events
[01.07 11:24:45] [e2m3u2b] Processed: 121031 events
[01.07 11:24:55] [e2m3u2b] Processed: 169640 events
[01.07 11:25:05] [e2m3u2b] Processed: 200224 events
[01.07 11:25:11] [e2m3u2b] XMLTVConverter: getDataTuple -> the empty value of 'title' is not allowed
[01.07 11:25:11] [e2m3u2b] XMLTVConverter: getDataTuple -> the empty value of 'title' is not allowed
[01.07 11:25:11] [e2m3u2b] XMLTVConverter: getDataTuple -> the empty value of 'title' is not allowed
[01.07 11:25:11] [e2m3u2b] XMLTVConverter: getDataTuple -> the empty value of 'title' is not allowed
[01.07 11:25:11] [e2m3u2b] XMLTVConverter: getDataTuple -> the empty value of 'title' is not allowed
[01.07 11:25:11] [e2m3u2b] XMLTVConverter: getDataTuple -> the empty value of 'title' is not allowed
[01.07 11:25:11] [e2m3u2b] XMLTVConverter: getDataTuple -> the empty value of 'title' is not allowed
[01.07 11:25:11] [e2m3u2b] XMLTVConverter: getDataTuple -> the empty value of 'title' is not allowed
[01.07 11:25:11] [e2m3u2b] XMLTVConverter: getDataTuple -> the empty value of 'title' is not allowed
[01.07 11:25:11] [e2m3u2b] Total processed 221190 events in 0:01:06.295 or ~3336 per second
[01.07 11:25:11] [e2m3u2b] [Favoris]: EPG data processing has finished 👍

I'd like to analyse and find which events have no title node. If I do a search of "<title></title>, I found nothing. So it seems that some events have no node <title>.

The problem is between 200224 and 221190 events , but in my xml I have a total of 43382 events (find with the unique value "<programme channel=".

So it's not easy to fix this epg file. Some events comes from rytec, some others from Pluto or Samsung and others from my own generation.

Have you an idea about how I can find the events causing these errors?

Thanks

Quote from jeepcook

I'd like to analyse and find which events have no title node. If I do a search of "<title></title>, I found nothing. So it seems that some events have no node <title>.

1) In your example, the title node in the "programme" element is present - RE: E2m3u2bouquet - discussion & support

2) The absence of a "title" node in the "programme" element or its presence with an "empty" value is complete nonsense and defies logic. Such elements of XMLTV are simply ignored by the parser code and are not imported. Because it doesn't make any sense

p..s. For a complete understanding of what is where

Снимок экрана 2024-07-01 в 19.33.48.png

those. If there is no "title" node in the "programme" element, then what is the logic to import it?

Quote from pepsik

Quote from jeepcook

I'd like to analyse and find which events have no title node. If I do a search of "<title></title>, I found nothing. So it seems that some events have no node <title>.

1) In your example, the title node in the "programme" element is present - RE: E2m3u2bouquet - discussion & support

2) The absence of a "title" node in the "programme" element or its presence with an "empty" value is complete nonsense and defies logic. Such elements of XMLTV are simply ignored by the parser code and are not imported. Because it doesn't make any sense

Yes in my example the title node is ok. But it's another case. I'm looking for other "programmes" which haven't the title, to understand why, to fix them or to ask the provider to fix them

Quote from jeepcook

I'm looking for other "programmes" which haven't the title, to understand why, to fix them or to ask the provider to fix them

In the xmltvconverter.py file, correct the exception output to display more detailed information about the “buggy” XMLTV element

Quote from pepsik

In the xmltvconverter.py file, correct the exception output to display more detailed information about the “buggy” XMLTV element

I think it's here I have to do the modification, but where and what...

def getDataTuple(now_timestamp_utc, elem):
    """here we get an event tuple of tuples or None

    ETSI EN 300 468
    0. start time (long)
    1. duration (long)
    2. event title (string)
    3. short description (string) -> 6.2.37 Short event descriptor
    4. extended description (string) -> 6.2.15 Extended event descriptor
    5. event type (byte - uint8_t) or list or tuple of event types -> 6.2.9 Content descriptor
    6. optional event ID (int - uint16_t), if not supplied, it will default to 0, which implies an
       an auto-generated ID based on the start time.
    7. optional list or tuple of tuples
       (country[string 3 bytes], parental_rating [byte - u_char]) -> 6.2.28 Parental rating descriptor
    """
    try:
        l = [get_timestamp_utc(elem.get(x, '').strip()) for x in ('start', 'stop')]
        if all(l) and (l[0] <= timespan) and (now_timestamp_utc <= l[1] > l[0]):
            l[1] -= l[0]
            if l[1] > 86400:
                raise Exception("an unacceptably large 'duration' value")
            l.extend(get_event_data(elem))
            if not l[2]:
                raise Exception("the empty value of 'title' is not allowed")
            while len(l[2].encode('utf-8')) > 220:
                l[2] = l[2][:-1]
            l.append(0)
            return (tuple(l),)

    except Exception as err:
        logger.debug("XMLTVConverter: %s -> %s" % (sys._getframe().f_code.co_name, err))
    return None

Quote from jeepcook

but where and what...

Quote from jeepcook

if not l[2]:

raise Exception("the empty value of 'title' is not allowed"

Depending on what you want to see in the console output... for example, the start time and end time of the "event"

            if not l[2]:
                print ("start: %s, stop: %s" % (elem.get('start'), elem.get('stop'))) 
                raise Exception("the empty value of 'title' is not allowed") 

Quote from pepsik

Depending on what you want to see in the console output... for example, the start time and end time of the "event"

Code

            if not l[2]:
                print ((start: %s, stop: %s) % (elem.get('start'), elem.get('stop'))) 
                raise Exception("the empty value of 'title' is not allowed") 

Thanks but the process hangs, I have no log because the process hangs before the end.

5002_0_1_D751_AACF_D596_DDDD0000_0_0_0_20240702081045.jpg

Quote from pepsik

in the console output

??? Look at system log.... Not at plugin log window

If you want to see the output in the plugin log window, then use logger.info instead of prin

            if not l[2]:
                loger.info("start: %s, stop: %s" % (elem.get('start'), elem.get('stop'))) 
                raise Exception("the empty value of 'title' is not allowed") 

Finally found the problem, I have a script which do a grep -v '<title>' to only keep the sub-title I rename title, but this doesn't work if there is no sub-title.

Sorry for the inconvenience.

Quote from jeepcook

I have a script which do a grep -v '<title>' to only keep the sub-title I rename title,

Leave both 'title' and "sub-title" ... the XMLTV parser uses them ... "sub-title" is interpreted by the parser code as a "short description"

Quote from pepsik

Quote from jeepcook

I have a script which do a grep -v '<title>' to only keep the sub-title I rename title,

Leave both 'title' and "sub-title" ... the XMLTV parser uses them ... "sub-title" is interpreted by the parser code as a "short description"

yes but for few channels the title is the name of the channel and the sub-title the real title. That's why I did that

Hello pepsik,

the arrangement in the playlist doesn't work. (see picture)

Everything was fine in version 1.7.0.Liste.jpg

Quote from hajosch

the arrangement in the playlist doesn't work. (see picture)

No log - no problems! No one here can guess from pictures.