pyTivo Discussion Forum Forum Index pyTivo Discussion Forum
Answers and the development of pyTivo a TiVo transcoding server
 
 FAQFAQ   SearchSearch   MemberlistMemberlist   UsergroupsUsergroups   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 

Possible workaround for "transfer failed" (bad dur

 
Post new topic   Reply to topic    pyTivo Discussion Forum Forum Index -> Support
 View previous topic :: View next topic  
Author Message
polarweasel



Joined: 06 Feb 2008
Posts: 9

PostPosted: Tue Feb 26, 2008 12:08 pm    Post subject: Possible workaround for "transfer failed" (bad dur Reply with quote

I'm using pyTivo to serve MPEG-2 movies from my PC to my S3 over a wired network. I recently noticed that a small subset of the movies on my computer fail to transfer to the S3 100% of the time; the S3 displays a generic "transfer failed" screen but doesn't provide any really useful information.

Here's my system information:

PC OS: Windows XP sp1
Tivo: S3 running latest software
pyTivo: The latest krkeegan snapshot from repo.or.cz as of about 3 days ago (I can't be more specific because I'm not at home)

As it turns out, the transcoding process is aborting because the movies all have "bitrate" values of "N/A" . They all also have "Duration" values of "N/A", which I expect is the root cause (ffmpeg can't calculate a bitrate for a duration of 0 milliseconds).

I turned on the debug output for pyTivo; here's the pertinent information:

video_info: ffmpeg output= FFmpeg version SVN-r11870, Copyright (c) 2000-2008 Fabrice Bellard, et al.
configuration: --enable-gpl --enable-pp --enable-swscaler --enable-pthreads --enable-liba52 --enable-avisynth --enable-libfaac --enable-libfaad --enable-libgsm --enable-libmp3lame --enable-libnut --enable-libtheora --enable-libvorbis --enable-libx264 --enable-libxvid --cpu=i686 --enable-memalign-hack --extra-ldflags=-static
libavutil version: 49.6.0 libavcodec version: 51.50.0 libavformat version: 52.7.0 libavdevice version: 52.0.0
built on Feb 5 2008 23:46:38, gcc: 4.2.3
Seems stream 0 codec frame rate differs from container frame rate: 29.97 (30000/1001) -> 59.94 (60000/1001)
Input #0, mpeg, from 'd:\Movies\Movie.mpg': Duration: N/A, start: 0.280633, bitrate: N/A
Stream #0.0[0x1e0]: Video: mpeg2video, yuv420p, 720x480 [PAR 32:27 DAR 16:9], 9800 kb/s, 59.94 tb(r)
Stream #0.1[0x80]: Audio: liba52, 48000 Hz, 5:1, 448 kb/s
Must supply at least one output file
video_info: Bug in VideoReDo
video_info: failed at kbps
video_info: Codec= mpeg2video width= 720 height= 480 fps= 29.97 millisecs= 0 kbps= None akbps= 448 acodec= liba52 afreq= 48000
video_info: d:\Movies\Movie.mpg cache hit!
video_info: d:\Movies\Movie.mpg cache hit!
tivo_compatible: d:\Movies\Movie.mpg max video bitrate exceeded it is None

The problem is occuring because video_info() is unable to process this ffmpeg output line properly:

Input #0, mpeg, from 'd:\Movies\Movie.mpg': Duration: N/A, start: 0.280633, bitrate: N/A

video_info() ends up converting a "bitrate" of "N/A" to "None".

Note the "millisecs= 0" and "kbps= None" entries. As I said, I think if my Duration value was something other than "N/A", ffmpeg could calculate a valid bitrate instead of dividing by 0, so I think the bad duration is the root cause of the failure.

Obviously, this is a problem with the MPEG-2 video files themselves, but I was wondering if there might be an intelligent workaround to this problem, i.e. some sort of intelligent default that could be used, but I don't know what decisions depend on the bitrate and thus could be screwed up by a default value.

One simple workaround for this problem would be to have video_info() return default values for the milliseconds and kbps variables if it can't determine the real values, but I'm afraid this would have undesirable side-effects in other areas. I'm willing to submit a patch if the folks who really know pyTivo agree on a fix.

Does anyone have a suggestion as to how to fix this properly? Or, failing that, how I can edit my MPEG-2 files to put in a valid dummy value for the Duration field?

Thanks ....

-Polar Weasel
Back to top
View user's profile Send private message
krkeegan
Site Admin


Joined: 04 Jan 2008
Posts: 412
Location: Los Angeles, CA

PostPosted: Tue Feb 26, 2008 3:46 pm    Post subject: Reply with quote

Another user is having similar issues where his duration for certain files is only returning as 5 seconds. And thus he only gets 5 seconds of video.

I have been thinking about this a little bit and I think I may have a solution that might be acceptable to everyone.

My Concerns:
-A default would be hard to set exactly. And would have to overestimate the longest file.
-The duration is used to calculate the estimated size and over estimating the duration could cause more shows to be delete from the TiVo than necessary.
-It would affect the asthetics of the play bar while the file is transfering to the TiVo(not a big deal I know)
-This would really be a hack/patch and not really a solution to the problem. (Which is ffmpeg not being able to determine the attributes of these files)

My Idea:
Instead of having a global default. Why not have a possible override for specific files.
For example a ".nfo" file or something that would contain all the information returned by ffmpeg.
Any attributes in this file would override the ffmpeg returned values.

Pros:
It gracefully solves the problem of bad files while allowing you to specify reasonable values instead of just default values.
It would not affect a user who does not have a ".nfo" file.

Cons:
It adds MORE random files to your video folder.
It requires makeing a file for each individual file.

What do ppl think?
Kevin
Back to top
View user's profile Send private message Visit poster's website
wmcbrine



Joined: 04 Jan 2008
Posts: 514

PostPosted: Tue Feb 26, 2008 4:07 pm    Post subject: Reply with quote

I don't think we need a new file -- this could be handled in the existing metadata files. I've been thinking along similar lines with regards to a "force reencoding" parameter -- something that wouldn't strictly be metadata, but could just as well be kept there. This would be an even better fit, since the duration actually is a metadata item.
_________________
My pyTivo fork
Back to top
View user's profile Send private message
krkeegan
Site Admin


Joined: 04 Jan 2008
Posts: 412
Location: Los Angeles, CA

PostPosted: Tue Feb 26, 2008 6:33 pm    Post subject: Reply with quote

Yeah agree that makes the most sense.

I think our only issues are duration and bit rate?? Is that correct?

Are there any other values that we get from ffmpeg that have been showing up improperly yet?

Or do we just want to allow all ffmpeg gathered settings to be cached just in case there is another issue in the future?
Back to top
View user's profile Send private message Visit poster's website
polarweasel



Joined: 06 Feb 2008
Posts: 9

PostPosted: Wed Feb 27, 2008 3:59 am    Post subject: Reply with quote

wmcbrine wrote:
I don't think we need a new file -- this could be handled in the existing metadata files. I've been thinking along similar lines with regards to a "force reencoding" parameter -- something that wouldn't strictly be metadata, but could just as well be kept there. This would be an even better fit, since the duration actually is a metadata item.


Since this is obviously an input-file-specific problem, I think an input-file-specific workaround is appropriate.

The workaround with the lowest impact to users would be to return some default duration, but I hadn't realized we'd have to overestimate the longest file -- I don't really care about the aesthetics of the play bar, but causing the TiVo to delete more shows than necessary would be a Bad Thing.

Adding something to the metadata file seems like a relatively low-impact, rational answer. Would the metadata addition look like

<Duration>3600000</Duration>
<Bitrate>8000</Bitrate>
(etc.)

or

<Force_Reencode>1</Force_Reencode>

I haven't mucked around with metadata files yet, but as I remember they're XML-based and would look something like this.

If users would need to add explicit values for the Duration, Bitrate, etc. to the metadata file, what would be the guidelines for entering usable values? They can't use ffmpeg to get the information in this case.

Would forcing a reencode with a flag in the metadata file result in the proper amount of space being used on the TiVo? If so, I think this is probably a better solution.

-Polar Weasel
Back to top
View user's profile Send private message
epsilondelta



Joined: 05 Jan 2008
Posts: 8
Location: Bloomington, Indiana

PostPosted: Wed Feb 27, 2008 5:20 am    Post subject: Reply with quote

For me, all the files on my PC that TiVo won't play have played OK in WMP or Winamp, or some other player. So you can get the duration & bitrate for such files from the player's info screen?

Also just me, but the ffmpeg command line and output are arcane, and so having a friendly list of what it thinks of a file in the metadata would be very helpful. Others will know better if this could/should include more than rate & duration.

OT: Love PyTivo, and what fun for a programming-challenged wannabe-nerd to be able to follow its development on this forum! Thanks to all the devs and posters here.

_________________
Ken
Back to top
View user's profile Send private message
PaulS



Joined: 04 Jan 2008
Posts: 148

PostPosted: Wed Feb 27, 2008 5:54 am    Post subject: Reply with quote

If you simply want to see details about a specific file, you can run it through MediaInfo and get output similar to ffmpeg. Check it out : HERE
Back to top
View user's profile Send private message
polarweasel



Joined: 06 Feb 2008
Posts: 9

PostPosted: Wed Feb 27, 2008 9:05 am    Post subject: Reply with quote

[quote="PaulS"]If you simply want to see details about a specific file, you can run it through MediaInfo and get output similar to ffmpeg. Check it out : [url=http://mediainfo.sourceforge.net/en]HERE[/url][/quote]

I have Media Info and have used it, but it doesn't report a valid duration for this file either. Here are all the utilities I've tried, and their reported durations and bitrates:

Utility Duration Bit Rate

Media Info v0.7.5.9: 26:30:00 340 Kbps

VLC v0.8.6d: 00:53:58 ~5000 kbps

WMP v10.00.00.3998: 00:00:00 Not Available

Winamp v5.33: 00:-3:-12 -168980 kbps

winamp v5.52: 00:00:00 70993951 kbps

Windows Movie Maker v5.2: 00:00:00.46 10448 kbps

Unless I'm going insane, the version of WMP I tried on my home system reported a duration of 01:50:00, which is probably approximately correct (I think that was probably WMP v9.x). The duration reported by VLC (00:53:58) is reasonable if we assume it's off by a factor of 2 because of the confusion between the "stream 0 codec frame rate and the container frame rate" (29.97 vs. 59.94).

It may be enough to guesstimate the duration that would go into the metadata file. If I do that, would I then compute the bitrate based on the file size and my guessed duration?

-Polar Weasel
Back to top
View user's profile Send private message
polarweasel



Joined: 06 Feb 2008
Posts: 9

PostPosted: Wed Feb 27, 2008 1:12 pm    Post subject: Possible solution: MPEG Streamclip Reply with quote

krkeegan wrote:
Another user is having similar issues where his duration for certain files is only returning as 5 seconds. And thus he only gets 5 seconds of video.
[snip]
Kevin


I found the thread discussing that user's problem and read through it. One suggestion was that the user try running the bad files through MPEG Streamclip to see if it fixed the duration.

I downloaded MPEG Streamclip from

http://www.squared5.com/svideo/mpeg-streamclip-win.html

and ran it on my movie file. Here's what I did:

Load the movie file with the bad duration into MPEG Streamclip
Click on Edit .. Fix timecode breaks (Ctrl+F)
Check "do not skip any frame" (not sure this is required)
Click proceed
(Wait for procedure to finish ...)
Click on File .. Save as ...
Save the movie to a new name

I then ran ffmpeg on both files, and the duration and bitrate appear to be fixed! Here's the pertinent output from the two ffmpeg runs:

E:\tmp\movie>ffmpeg -i movie.mpg
[snip]
Input #0, mpeg, from 'movie.mpg':
Duration: N/A, start: 0.280633, bitrate: N/A
Stream #0.0[0x1e0]: Video: mpeg2video, yuv420p, 720x480 [PAR 32:27 DAR 16:9], 9800 kb/s, 59.94 tb(r)
Stream #0.1[0x80]: Audio: 0x0000, 48000 Hz, 5:1, 448 kb/s

E:\tmp\movie>ffmpeg -i movie1.mpg
[snip]
Input #0, mpeg, from 'movie1.mpg':
Duration: 01:30:23.5, start: 1.000000, bitrate: 5873 kb/s
Stream #0.0[0x80]: Audio: 0x0000, 48000 Hz, 5:1, 448 kb/s
Stream #0.1[0x1e0]: Video: mpeg2video, yuv420p, 720x480 [PAR 32:27 DAR 16:9], 9800 kb/s, 59.94 tb(r)

I've now tested this with two MPEG-2 files with bad durations and bitrates, and it's fixed both of them -- both files now transfer to my S3.

I suggest that users do this instead of adding additional fields to the metadata (this will keep things simpler).

-Polar Weasel

[Edit 28-Feb-2008 03:08 UTC to reflect additional necessary step to fix duration and include the fact that this works!]
Back to top
View user's profile Send private message
Display posts from previous:   
Post new topic   Reply to topic    pyTivo Discussion Forum Forum Index -> Support All times are GMT - 8 Hours
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You can download files in this forum
Site is in NO WAY affiliated with TiVo Inc

Powered by phpBB © 2001, 2005 phpBB Group
phpBB SEO
[ Time: 0.3266s ][ Queries: 13 (0.2347s) ][ Debug on ]