Chocholousek Picons updater (Enigma2 plugin)

No, nothing arrived, sorry.

Try the following:

1. click on my nickname (under my avatar / my icon)

2. then click on "User" - i.e. a stick figure icon

3. where the context menu will appear... please choose the option: Create Conversation

4. put the name of the subject in the first field (for example "chocholousek picons plugin")... and then below, in the large field, you can write a private message... and of course it is also possible to attach attachments in this private conversation

Hi viliev

First of all, it should be noted that you are using OpenATV version 7.4, which is a test version intended for developers. Still, this version is not finished and is still under development. That is that errors may still appear randomly in Enigma2 during regular updates of this beta or alpha version. You mentioned two set-top boxes, so I assume that ATV 7.4 is installed on both of them.

Secondly, regarding the BUG, as you can see, the following problem occurs:

06:52:12.3825   File "/usr/lib/enigma2/python/Plugins/Extensions/ChocholousekPicons/plugin.py", line 1540, in proceedCfgProfile
06:52:12.3862     bq_contents += f.read()
06:52:12.3866                    ^^^^^^^^
06:52:12.3867   File "<frozen codecs>", line 322, in decode
06:52:12.3868 UnicodeDecodeError: 'utf-8' codec can't decode byte 0xd0 in position 49451: invalid continuation byte

ChocholousekPicons source code: https://github.com/s3n0/e2plug…ugin.py#L1540C44-L1540C44

Enigma2 source code crashes for a partially known reason. However, since the ChocholousekPicons plugin runs at that moment only in a separate thread (i.e. a separate algorithm in the background), the entire Enigma2 (ATV 7.4) will not crash. Under normal circumstances, however, Enigma2 should crash (BSOD / GSOD will appear). Then subsequently, in the event of an error in the execution of the source code in the ChocholousekPicons plugin, it will terminate regularly + also with the error reported by the plugin: "No operation! Picon update failed!".

In the debugging information (debug-log file) it can be seen that the problem arises when reading the files "/etc/enigma2/userbouquets.*.tv" (userbouquets.*.radio) in connection with the decoding of UTF-8 characters. Specifically, there is a problem with decoding some character 0xD0.

You can try to check your userbouquest files to see if their encoding is correct and if, for example, some other plugin has written some wrong characters into them. If these userbouquets files are fine, try reporting a bug on the opena.tv discussion forum. Although... at this moment, ATV 7.4 is still under development. So you can... but you also don't have to... report this BUG. The choice is only yours. If this is really a bug of ATV 7.4, then probably the developers just didn't notice this bug and will fix it soon.

You skipped the first step - check your files "userbouquet.*.tv" and "userbouquet.*.radio". There is probably an error in them, which Enigma2 cannot deal with, because it received an over-the-air update.

I have OpenATV 7.3 only and I have absolutely no problem with myself.

I can also... if I have free time... test the development version ATV 7.4... via OpenMultiboot... on my Vu+ device. And I'll see if the error manifests itself in me as well.

But first we will test your userbouquest files. If you can't manage it yourself (you haven't written anything about your IT skills yet), you can send them to me (preferably as one zipped file, of course).

Hi viliev

I couldn't find the OpenATV 7.4-devel version for the Vu+ Solo SE V2 anywhere on the internet, so unfortunately I can't test it myself.

If you want, you can create a BUG report for this "7.4-devel" version on the OpenATV discussion forum. You can also attach all the URLs what we discussed together + including of the CODE TAGs contents. You can also attach your userbouquet files to the OpenATV forum.

It seems to be really just a problem with the ATV 7.4-devel development version.

The error will probably be in the "UTF-8" decoding of your userbouquet files... however, the error may not be directly in the file but in the algorithm that reads the data from the file. I tried it now in OpenATV 7.3 @ Python 3.11.2 - in the official released version, and everything went OK. I simulated the activity of my plugin with your userbouquest files (copied into the "/tmp/u" folder):

root@vusolose:~# python
Python 3.11.2 (main, Feb  7 2023, 13:52:42) [GCC 12.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import glob
>>> bouquet_files = glob.glob('/tmp/u/userbouquet.*.tv')
>>> bq_file = bouquet_files[0]
>>> bq_file
'/tmp/u/userbouquet.samsungtvplus_us.tv'
>>> bq_contents = ''
>>> with open(bq_file, 'r') as f:
...     bq_contents += f.read()
...
>>>
>>>#........all done ? ..... no error occurred

Btw, what version of Python does this ATV 7.4-devel use ? Please connect via Telnet / SSH to your set-top box and type the command "python --version".

If you could manage it... you can try the same... after connecting via Telnet / SSH to your set-top box. So, first you start the Python interpreter (write the command: python). Unpack / copy the userbouquets files into a temporary folder in the set-top box "/tmp/u" or change the name of the directory path to your own, i.e. from "/tmp/u" to "/etc/enigma2". And then proceed in the Python interpreter, according to the above steps.

OK, so please try this in both cases, in the Python interpreter, as I mentioned above:

import glob
bouquet_files = glob.glob('/etc/enigma2/userbouquet.*.tv')
bq_file = bouquet_files[0]
print(bq_file)
bq_contents = ''
with open(bq_file, 'r') as f:
  bq_contents += f.read()        # an error occured ?!
chardet.detect(open(bq_file, 'rb').read())['encoding']

@zephy4

Yes I know. The plugin basically works everywhere and for everything. No one has reported this error. Only viliev reports a minor problem.

I'm trying to find out exactly what the problem is. So far we have only found out that it is a problem with detecting UTF-8 in userbouquet files - specifically in loading content from these userbouquet files.

The question now is whether this is a BUG:

- OpenATV 7.4-development

- Python (BUGs in Python itself are not unusual and sometimes happen)

- wrong format of userbouquets files

- an error in the source code of the plugin

I can use UTF-8 to load content from a file, but that's not a problem, since files are loaded by default as UTF-8. The problem is that the decoder sees the character 0x0D there, which is used when formatting Windows newlines 0x0D + 0x0A (CR + LF). Whereas for UNIX only character 0x0A (LF) is used.

The error can really be anywhere. Probably userbouquet files are edited by some plugin and put wrong character in formatting other than UTF-8.

Exactly Warder 👍

I didn't give it a chance at first. But you're already the second person to confirm it to me. Namely... that there really could be a problem in the userbouquet files.

I had all the "userbouquet.*.tv" files supplied by viliev checked in a loop (I have them in a private message as a ZIP archive) and indeed there are some files with the wrong format: ISO-8859-1, MacRoman, Windows-1252, ASCII, ... etc. .

import glob
bouquet_files = glob.glob("/tmp/u/userbouquet.*.tv")
for PATH in bouquet_files:
  print("%s ---- %s" % (chardet.detect(open(PATH, "rb").read()), PATH))
  with open(PATH, "r") as f:
    data = f.read()             # an error occured ?!

//////////// EDIT :

Hm, that's weird. When I open the files individually on the PC, they look fine in Notepad++. Maybe a BUG in Python 3 ? :-D

It is actually my fault that my plugin does not write the specific userbouquest file it is currently processing, in the debug log. I don't want to create an unnecessarily large "FLOOD" in the logs - while debugging Enigma2.

I discovered that the file "userbouquet.ciefpsettings_filmovi3.tv" could be faulty.

My conclusion:

I don't know if the problem is in the encoding of UTF-8 characters, but definitely these userbouquet files are not correctly encoded in UTF-8 ! As you can see on the attached screenshot, they contain meaningless characters, especially the 0xD0 character, which causes a BUG in Python, and which the UTF-8 decoder tries to decode in every possible way... but it doesn't always succeed. It is necessary to repair / edit these strange userbouquet files... or delete them altogether.

>>> bq_file = "/tmp/u/userbouquet.ciefpsettings_filmovi3.tv"
>>> with open(bq_file, "rb") as f:
>>>   data = f.read()
>>> type(data)
<class 'bytes'>
>>> data.decode('utf-8')
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xd0 in position 49451: invalid continuation byte
>>>

Quote from viliev

Would you like to tell me how to make "loop-test-output.txt"

Hi.

Use the first code (at the top) from my post #372 :

Post

RE: Chocholousek Picons updater (Enigma2 plugin)

Exactly Warder 👍

I didn't give it a chance at first. But you're already the second person to confirm it to me. Namely... that there really could be a problem in the userbouquet files.

I had all the "userbouquet.*.tv" files supplied by viliev checked in a loop (I have them in a private message as a ZIP archive) and indeed there are some files with the wrong format: ISO-8859-1, MacRoman, Windows-1252, ASCII, ... etc. .
(Code, 7 lines)


//////////// EDIT :

Hm, that's weird. When I open the files…

s3n0

Dec 17th 2023

However, as first, edit the path to the directory in the 2nd line of code, to "/etc/enigma2/userbouquet.*.tv".

You can first copy the entire code into a text editor on your PC. Then edit the mentioned directory. And finally, copy the text again in the text editor, to the Windows clipboard, using the CTRL+C keyboard shortcut.

1. Connect via Telnet / SSH protocol to your set-top box

2. Then start the Python interpreter - command: python<ENTER>

3. Next... you need to copy text... by marking text with the help of the mouse and using the keyboard shortcut CTRL+C... of the entire code

4. If you are using the PuTTY application as a terminal for Telnet / SSH connections, then, you just click on the PuTTY window with the mouse - with the right button ... and the content from the Windows system clipboard will be inserted (correctly: "sent") into the PuTTY terminal. (If nothing works, you can copy the lines one by one into the PuTTY terminal or you can rewrite the whole thing manually via the keyboard.)

Quote from merouane1

Merci pour la mise à jour. Jais un problème des picons sur le Sat Nilesat ( 7°W) la chaine Algerien mais le picone ne correspondant pas à la chaine et il y a des chaines non pas des picone et merci.

Hi. Try it here: Picons by Chocholoušek

Hi cpr43 .

This is more of a question for the creator and designer of the picons - a question for chocholousek . I am just a plugin developer for these beautiful picons from Chocholousek, which are available in all the most popular image resolutions.

I really don't know if Chocholousek has the desire and free time to create picons even for satellites on the North American continent (USA). That would probably also mean reworking the plugin for changing the US / EU locations.

What are the service reference codes and standards in the USA for satellite reception ? Are they the same as in Europe ? Does this mean that DVB-S2 is used there for satellite TV reception ?