LinHES

OK, I have the problem too, now. Attached is the log of my attempt to run mythtranscode (which is almost certainly the cause of the problem). Here is the log file. I'm investigating it now and will give more results later.

Mike

Hmm... Looks like it's a problem with libraries.

A recompile would probably fix it but obviously there are incompatible libs installed, which should be impossible. The output log doesn't actually tell you which one it barfed at, but I suspect the last one would be a good bet. Anyone?

Mike

EDIT: Last library was ld-2.9.so from package glibc 2.9-4 (makes sense, since it's a glibc error). Cecil?

Here's the output from reinstalling glibc, which didn't fix the problem

I think it's our good friend, mjpegtools. mpeg2enc (which is part of mjpegtools) appears (to me) to the the failing puzzle piece as evidenced with INFO: [mpeg2enc] Encoding MPEG-2 video to /myth/md0/archive/work/temp.m2v and then /myth/md0/archive/work/temp.m2v comes up as a zero byte file for me. Opinions?

Are all of the affected using the vdpau version of mythtv perchance?
Here are my versions:
[mythtv@mythtv mytharchive]$ pacman -Q|grep myth
mythappletrailers 0.04.3-29
mytharchive 0.21-8
mythbrowser 0.21-6
mythcontrols 0.21-4
mythdb-initial 1-42
mythflix 0.21-5
mythgallery 0.21-5
mythgame 0.21-4
mythinstall 1-10
mythmovies 0.21-5
mythmusic 0.21-12
mythnews 0.21-4
mythphone 0.21-3
mythsmolt 0.21-8
mythstream 0.18_1-8
myththemes 0.21-3
mythtv-vdpau 0.21-23
mythvideo 0.21-9
mythvodka 0.7-11
mythweather 0.21-3
mythweb 0.21-20

It doesn't seem altogether stupid that there's a version mismatch problem.

I attempted to compile the latest stable code for mythplugins. After a few false starts it seems to be going along. More later.

Well, gosh darn it...

Anyone who is experiencing this not using the vdpau version? Anyone?

There's a good chance it's a combination of libjpeg and mythtv-vdpau.

Mike

I'm not using the VDPAU on my MBE where I ran into trouble with MythArchive...[root@mythbox-mbe ~]# pacman -Q -o `which mythtranscode` /usr/bin/mythtranscode is owned by mythtv 0.21-81

Hi Mike,

Thanks for your feedback: That takes mythtv-vdpau out of the equation for now.

Didn't realise it was the menu video creation that was failing. The mjpegtools have been flagged as a problem previously, so you may be right there. Determining that mpeg2enc is what creates the menu video is not easy from that script.

Have you re-compiled mjpegtools? I'll do it now. I'm working remotely so won't be able to test it until tonight.

Debugging the script can be enabled with "debug_keeptempfiles = True". Will do this tonight, too.

Mike

I recompiled mjpegtools locally and had the same results (as I expected).

Me too, as it happens.

I also attempted to generate an iso without a menu (the simple option) and this failed too. It failed immediately, in fact. There seemed to be something preventing it from trying, so I deleted the contents of /myth/tmp/work and tried again. It still failed. Seems that something is caching the fail result and stopping it from trying again.

What do we do next? This looks like a major problem which will affect everyone. A show stopper if you will. Should we mod this up to priority one?

I recompiled mjpegtools locally and had the same results (as I expected).

On a hunch I also recompiled imagemagick and transcode but once again, same result.

Do you think glibc may be responsible?

Can't say I know enough about it. Give it a try?

I just tried
mplex -f 8 -o output.mpg input.m2v input.ac3
manually and it worked perfectly using the original version.

This problem totally screws up any archiving for me. Is this the same for you?

Replacing glibc is daunting, since I know how many things it can break (the whole system) and I'm testing on a prod system.
There is a new version: 2.9.4 is the pacman version, 2.10 is the latest.

Cecil? Any ideas or suggestions?

Mike

I'm able to create an ISO w/ "Simply/auto play". However, I cannot create an ISO w/ a menu. It seems the issue maybe w/ page generation in mythburn.py.

Looks like an [[http://svn.mythtv.org/trac/ticket/6981|upstream issue]].

That's probably good news. There's already a fix for it, so Cecil, is it easy enough to diff the file and re-compile replex?

Mike

I wish is were that easy. I don't think it is the same issue. Additionally the patch is against trunk.

Is it better to wait until the end of the month for 0.22 then?
EDIT: I've downloaded trunk and am compiling it now with the patches applied. Will report level of success of course.
At least we should have an idea of whether or not 0.22 will break anything/everything.
EDIT2: Installed it all in /myth/local. It definitely fixed mythtranscode (woo-hoo!). Will test the rest later.
EDIT3: No, doesn't work. Might be doing something wrong, but tried last night and it failed without telling me why. May have to work out how to do DVD menus manually

OK, someone in the UK has suggested it may be the vdpau version. Post is at http://www.knoppmyth.net/phpBB2/viewtopic.php?p=123808#123808

Now I think it only affects the DVB users and Mike Hanson's problem is not related.

I'll revert as soon as I can and try again.

Mike

FWIW, the recordings I'm attempting to archive are ATSC, recorded with tuner cards setup as DVB cards. I've launched the mytharchive process from frontends that have the mythtv-vdpau package installed as well as from the backend itself, which does not have the mythtv-vdpau package. Results are the same no matter where I start the mytharchive process. Cesman has the problem too, but said it did not happen when "Autoplay - Simple" profile was selected in mytharchive (no menus). I have not tested the "Autoplay - Simple" method yet, but I'd be willing to bet that if cesman said it worked, it will. The problem seems to lie in the menu creation.

I backported to mjpegtools-1.8.0-3.pkg.tar.gz which gave me the following error:

Which I think is where the error lies as jpeg2yuv is what is used to create the m2v file. So I tried running the command from a shell and got the following dependency error:

So I forced pacman to install libjpeg-6b-5-i686.pkg.tar.gz by ignoring dependencies. Running the jpeg2yuv command above from a shell worked that is to say it generated a non 0 length temp.m2v file however forcing the libjpeg-6 install made it so that mytharchive would error early on with the following error in mythburn.log:

I am guessing that imports in python do not like the older version?
This may or may not be helpfull. I don't know if I have actually discovered anything other than the fact that it used to generate the menus with the older packages, which we already know since mytharchive used to work :-) But, maybe this is useful somehow.

I forgot to include in the last post that it appears that there is specifically a problem with jpeg2yuv. If you run:

jpeg2yuv -n 447 -v2 -I p -f 29.97 -j '/myth/tmp/work/background-1.jpg

from a shell it will segfault.

That last few posts go along with my suspicion that the libjpeg update is to blame.

Well, I found a solution. It however means recompiling various files. I shall endeavour to get this resolved this weekend.

Jams and I have colaborated on a solution.

Should drag all you need. Additionally, you can select Service Menu > Upgrade LinHES.

I tried the upgrade and mytharchive is still failing early on with the last batch of errors that I posted above from mythburn.log on lines 72, 15 and 135. I hope I haven't hosed something trying to get it to work.

I had an error that was in /data/var/cache/pacman/pkg/.INSTALL when I first upgraded it said unrecognized command 'qrep' is the command supposed to be 'grep'?

I just upgraded it and it went fine:

Which tells me Cecil was REALLY onto this, compiling it only a few hours ago!
I'm not at my machine yet to try the archiving but there was no .INSTALL file in that package, either:

Can you paste your upgrade command and errors. If it's too long, gzip it and attach as a file. Also, do an:

This file doesn't (and shouldn't, AFAIK) exist on my machine. It looks like someone or a pkg process has maybe unpacked a package in the pkg cache directory, and this could be causing pacman to barf.

Mike

Depending on when you download you've have encountered the qrep error. This has already been corrected. I've tested the upgrade on two machines w/o issue. I've also tested on a clean install w/ a new ISO.

I believe I unpacked a package in the pkg directory. My bad. I still can't get past the python errors saying that I don't have _imaging installed. If I try import _imaging at a python prompt it says it can't find libjpeg.so.62. I know that file is now in /opt/libjpeg6/lib/ why is python not finding it? I tried adding it to my PYTHONPATH but that didn't work.

Make sure 6b-10 is installed.

Ok, for some reason the library path in /etc/ld.so.conf was set to /opt/libjpeg6/include and didn't have /opt/libjpeg6/lib listed. I added it and jpeg2yuv worked but produced a garbled and mostly green yuv file. I manually added using pacman -U mjpegtools-1.8.0-3.pkg.tar.gz and the yuv output looks correct. I have tested it with mytharchive and it produces the correct menus and all seems to work. Has anyone else run into the green garbled screen problem? Is the newly compiled mjpegtools-1.9 package supposed to use libjpeg-6 or have I just created my own library mess?

You should have:
pil 1.1.6-11
mytharchive 0.21-10
mjpegtools 1.9.0-9 (or -10, -10 doesn't appear to be sync'n but the version was just bumped for a recompile)
libjpeg6 6b-10

I'm going to bump libjpeg to -11 and fix the ld.so.conf. When libjpeg6 is installed, it does symlink, but ld.so.conf should be corrected.

I checked all my packages and all were up to date except for libjpeg which after updating to -11 everything runs without an error but it is producing green blank menus again. Something is still amiss for me. I ran pacman -U mjpegtools-1.8.0-3.pkg.tar.gz and it produces good menus with that version.

Please remove all the packages and reinstall. I've tested multiple times on multiple systems after jams and I resolved the problem. Is there a particular theme you are using?

I removed all the packages and re-installed and I still get the green screened menus. I know that it doesn't have anything to do with the theme since I can create a test yuv file that looks green (when played with mplayer) too using:

If I use mjpegtools-1.8.0-3 the generated file and menus look correct. The only thing that I have noticed that is different is that when I run the 1.8 version of jpeg2yuv with verbose output at 2 it outputs the following:

where as if I run the 1.9 version I get:

Which may have something to do with it.

I did the same thing on my local 32 bit Ubuntu machine with a single frame and it worked beautifully.
I repeated this on my LinHES box (same jpg) and got a green mess like Nathan.
Screenshot attached.
Got the exact same "INFO: [jpeg2yuv] Rewriting latest frame instead." with both machines, so it's not relevant to the problem.
LinHES box compiled from source version failed with a seg fault, even after I re-compiled it.
Compiled new mjpegtools on remote 64 bit Ubuntu box and yuv was OK.
Looks quite specific to LinHES. The compilation went flawlessly, but the binary didn't work - very odd. I'll have a look at what libs jpeg2yuv uses next.
EDIT: Um, sorry everyone (especially Cecil). My jpeg2yuvs were linked to the recompiled libs. I removed the path from the ld conf, re-linked and the package version of jpeg2yuv works beautifully. The recompiled one now goes all the way through but now it produces green mess. Conclusion is that the compiled libs are the problem, NOT the package ones.
Nathan, what does ldd `which jpeg2yuv` tell you?

Here is my output from ldd `which jpeg2yuv`

I think you have compiled mpegtools from source with no configure options. Your libs should be in /usr/lib, not /usr/local/lib.

You can fix this by either removing the compiled version (in the source directory, do "make uninstall") or remove /usr/local/lib from your library path in ld.conf (then you must run ldconfig).

Here is my (now) working ldd with all versions as Cecil suggested: Yours should be the same.

mike@mythtv:~$ ldd /usr/bin/jpeg2yuv linux-gate.so.1 => (0xb7fbe000)
libmjpegutils-1.9.so.0 => /usr/lib/libmjpegutils-1.9.so.0 (0xb7f9b000)
liblavjpeg-1.9.so.0 => /usr/lib/liblavjpeg-1.9.so.0 (0xb7f76000)
libjpeg.so.62 => /usr/lib/libjpeg.so.62 (0xb7f56000)
libm.so.6 => /lib/libm.so.6 (0xb7f30000)
libpthread.so.0 => /lib/libpthread.so.0 (0xb7f18000)
libc.so.6 => /lib/libc.so.6 (0xb7dd1000)
/lib/ld-linux.so.2 (0xb7fbf000)

Thanks mike, You were right on. I did compile mjpegtools at one point early on so, doing the make uninstall did the trick. Sorry for leading us down the beaten path. This is where my lack of skills gets me in trouble. However, I am glad to learn about the ldd trick though, I kept thinking "If only I could see which libraries were being used.."

No problem Nathan: I knew about the compiling only because I'd made the same mistake.

Did this fix everything? I'm not at my LinHES box ATM.

Mike

P.S. ldd isn't a "trick" but a debug tool. Linux/Unix is full of them :-)

I upgraded via -Syu last night and now I get this:

Wonder why spumux is looking for the old library. Have you tried creating a symlink? I know it rarely works, but worth a try.

Have not tried adding the link. That normally will break something else, but I suppose I could try and then remove the link after.

Looks like dvdauthor needs a recompile. I just used the LinHES PKGBUILD with makepkg and installed. ldd reports the correct usr/lib/libMagickCore.so.2 now. Previously ldd indicated usr/lib/libMagickCore.so.2 was not found. I'll run MythArchive thru it's paces and see what happens.

Cool. Well, we're getting there.
The symlink was only suggested as a temp debugging thing to find the cause of the breakage.

Any idea how to run it from the command line, Mike? That way I could run it remotely (not necessary now, since I'm home, but would be useful).

Mike

Every thing seems to work fine for me now. I haven't had any issues but, I will run a few more tests. I haven't tested out any of the animated themes. The only issue that I have noticed that is somewhat unrelated is that the Internal players highlight box does not show up when previewing the DVD but, it has issues with all the DVDs I play. Everything looks fine with Xine.

Animated themes tested fine here. :)

After a recompile bump of dvdauthor, Mythcenter - Animated works for me. :)

I did a successful authoring job too. Even with the menu :-) Didn't burn it to DVD yet, but that's not the broken bit.

Well done, guys. Think you fixed it.

Mike

EDIT: Burned a DVD just to be certain, and it worked beautifully, too.