pyTivo Discussion Forum

rdian06 · Joined: 12 Apr 2008 Posts: 362

I've got a clip that is just under 60 seconds long that when I transfer with pyTivo, my S3 thinks it's already played and then asks me to keep or delete it. Keeping it doesn't help and I could never play it back before.

But using the "Save to VCR" option causes the S3 to display that the duration as 3 minutes and then the video starts playing from the start with a progress bar that clearly shows just under 60 second duration.

Sending metadata didn't help.

I think I have an even shorter (smaller) clip that also does it. wmcbrine, PM me how you want me to get the clip to you.

rdian06 · Joined: 12 Apr 2008 Posts: 362

TeamSpring5 · Joined: 07 Aug 2008 Posts: 1

I'm seeing the exact behaviors discussed in this thread.

I'm new to pyTivo and let me say this is a great tool. I'm using KRKeegan's fork, S2 with the default pyTivo.config that is created with the windows install (running XP SP3).

Thanks for developing this product - we have invested countless hours producing home movies and it is great to have them available on our Tivo. Thank you, Thank you.

rdian06 · Joined: 12 Apr 2008 Posts: 362

rdian06 · Joined: 12 Apr 2008 Posts: 362

I switched to wmcbrine's latest fork and the problem is still there.

I noticed that with the clips I'm sending back, I seem to be consistently dropped in at the 7 minute mark on a long recording (greater than 30 minutes), but things shorter seem to always get the stuck at the end upon trying to replay.

I used ffmpeg to create tivo compatible mpeg clips of various lengths to test with and tought I finally found a 7 minute clip that transfers with replay working correctly, but a subsequent transfer got stuck at the end on replay.

Looking over the metadata code, I see there are some fields for when the recording was originally made that are based on the current time and duration of the clip. My hunch is that the Tivo's replay craziness is related somehow to those values. Because the only thing that should be changing between transfers of the same file are those timestamps.

rdian06 · Joined: 12 Apr 2008 Posts: 362

Good news. I think I'm very close to solving this bug.

It involves the time, startTime, and stopTime metadata values. I just need to do a few more tests to completely understand what's going on so I can fix the code properly.

rdian06 · Joined: 12 Apr 2008 Posts: 362

The good news is that I've found the source of the problem. The bad news is that I'm not sure how to properly fix it in pyTivo or even if it should be fixed in pyTivo.

The cause of this great annoyance is lack of time synchronization between the computer running pyTivo and the Tivo box AND the way pyTivo sets the "time" metadata value.

Short version: If the clock on the computer running pyTivo reads a later time (it is out of sync with Universal time), then recordings transferred to your Tivo (which is synced to Universal time when it calls in to Tivo.com) will have a playback start point offest from the beginning of the recording equal to the amount of drift. If the amount of clock drift is greater than or equal to the recording length, then the playback cursor is STUCK at the end of the recording when you try to replay it. If you try to play it as it's transferring, the playback cursor will automatically advance to the latest point in the transfer.

For example, my pyTivo computer's clock runs 5 minutes fast (because it's old and Windows XP's time synchronization breaks easily.) When I transfer a recording from pyTivo to my Tivo, the playback start point will be:

- 5 minutes into the recording if I didn't try to play it until the transfer was complete and the clip is longer than 5 minutes
- stuck at the end of the recording if I didn't try to play it until the transfer was complete and the clip was 5 minutes or shorter
- the farthest point into the recording if you tried to play it before the transfer completed and less than 5 minutes of the recording have transferred so far
- 5 minutes into the recording if you tried to play it before the transfer completed and more than 5 minutes has been transferred

Long technical version:

Your Tivo's clock is set by Tivo.com and the timezone is auto selected based on your zipcode (Guide data setup.) The clock on your computer running pyTivo can drift either backwards or forwards depending on the quality of the hardware clock on it's motherboard and whether your BIOS battery is still holding a charge (and for those of you who have Intel Macs and dual boot between Mac OS X (assumes BIOS time is Universal Time) and Windows (assumed BIOS time is localized to the time zone you set in Windows). Older computers tend to develop clock drift as they get older.

On your Tivo, when you navigate a pyTivo share and select a video to view it's description and are offered the transfer option, your Tivo is making a call to pyTivo which in turn collects metadata about the file to send back to the Tivo. pyTivo populates a "time" metadata field by querying the pyTivo computer for the current time. The "time" metadata field is supposed to be set to the time the Tivo recorded the clip (i.e. you're using MRV and transferring from one Tivo to the other so this field represents the original record time). (The "startTime" and "stopTime" metadata fields are completely useless for transfers as far as I can tell.)

Now once you choose the Transfer option and the video begins to transfer, the Tivo seems to calculate the difference between the "actual time when the transfer started" and the reported "time" metadata field from when you originally navigated to the description to define where "playback should begin relative to the beginning of the file". If the Tivo clock and pyTivo computer clock are synched, then the "time" field should be in the past relative (a negative difference) to the "actual time when the transfer started" and the Tivo will begin playback at the beginning of the recording. The exact value of the negative difference is equal to the time you sat at the description screen before actually choosing Transfer (and the transfer actually beginning assuming another transfer isn't in progress.) As long as the difference is negative or zero, playback will always begin at the start of the clip regardless of whether you are playing it after the transfer completed or while the transfer is in progress.

If your pyTivo computer's clock is running fast, then there is a possibility that the "time" metadata field is in the future relative to the "actual time when the transfer started" (a positive difference.) The actual difference will be the value of the drift minus how long you sat at the description before choosing Transfer (and minus whatever time it took to finish any pending transfers before finally beginning this one.) If there is a positive difference and you view the file after the transfer is complete, the playback start position will be set to that positive value. If that positive value is beyond the end of the clip, then you're stuck at the end of the clip and can't rewind. If there is a positive value and you begin playback while the transfer is in progres, playback position will be set to the postive time from the start of the clip (assuming that many minutes or more has been transferred) OR playback position is set to the farthest point of the transfer (assuming fewer minutes than the positive value has been transferred.) If the postive value is beyond the length of the clip and you start play before the transfer finishes, then you start at the farthest point and can rewind. But once you finish playback (and the transfer) and try to play it again, you're stuck at the end of the clip and can't rewind.

The Tivo seems to do this so that it can properly set the start playback point for recordings longer than the actual record definition. Why would you ever have a recording longer than the record definition. Say you tune to channel 2 at 7:15PM and are just watching live TV. Then at 7:35PM, you decide you want to record the half hour current program that began at 7:30PM on channel 2 and runs until 8PM. Rather than starting the recording at 7:35PM and losing the first 5 minutes of the program, your Tivo will actually use what's in the live buffer and continue on. So you end up with a recorded clip that runs from 7:15PM to 8PM, but the perpetual start point is set to 7:30PM. So every time you restart playback of that clip, it will start at 7:30PM and you can actually rewind back to 7:15PM, but it will NEVER just start playing back from 7:15PM without you rewinding it first.

It turns out my pyTivo computer has been accumulating drift for months now. When I first got my original Tivo S3 four months ago and started using pyTivo, the drift was barely noticable and was probably cancelled out by navigating through the menus. Over the months, the drift has accumulated to around 8 minutes currently. The firewall config on the box prevented it from synching time with the built in Microsoft time servers (not to mention that Microsoft's time servers are over taxed and will not return a reply a good deal of the time.) And it didn't always bite me because I would queue up a bunch of transfers in succession. Since the tranfers tended to be for 30 to 60 minute recordings, only the first transfer would be screwed up. All the subsequent ones would have "caught up" with the metadata by the time the actual transfer started resulting in negative time differences.

Also, some people don't bother setting their Windows computer clocks to the correct timezone. They just adjust the time and leave the timezone set at the default time zone on install (Microsoft in Redmon set's it to Pacific for installs from retail or volume media, Dell based in Texas sets it to Central for their factory images.) So when pyTivo asks for the time, it takes the incorrect timezone into account and ends up with the wrong Universal time, potentially hours off.

Possible fixes:

1) Make sure your pyTivo computer has it's time synched to Universal time via the Internet. Google for a time server that doesn't suck and configure that in XP's date and time settings.

2) Modify the pyTivo\video\plugins\video\templates\TvBus.tmpl to remove the two locations that specify the "time" field. If you omit transmitting the "time" field in this way, the Tivo will auto set the "time" field to the "originalAirDate" which pyTivo will send as the "create time" of the video clip from your pyTivo computer's file system. Unfortunately, if you have wacked out file system timestamps in the future, then you'll run into the same problem. But the likelihood of that is probably smaller. The second drawback is that if you sort your Now Playing List by date, the current pyTivo logic has new transfers always at the top. This modified version would have new transfers inserted potentially anywhere in the list. I don't really like this solution. I just tested to see what the Tivo would do.

3) Probably the RIGHT way to fix this is periodically have pyTivo query Universal time directly via the web and somehow notify the user (maybe send an email) with an alert to fix their computer time. At the very least we need to document this in the FAQ.

4) Maybe we can get pyTivo to query the Tivo for the proper time and use that to fill in the "time" field rather than trusting the computer's clock. I haven't investigated this at all.

wmcbrine · Joined: 04 Jan 2008 Posts: 444

Well, that would explain why I never saw it. (I run ntpd on my system.)

TiVo's own MRV transfers give the transferred programs the timestamp from when they were recorded, which means they can indeed appear anywhere in a date-sorted list. But I've always felt that was (by definition) the correct behavior, and this is one more reason to think so.

Thanks for your efforts on this.

rdian06 · Joined: 12 Apr 2008 Posts: 362

Yeah, I just have to force myself to pass metadata so my files are easier to find. I'm really bad at naming stuff appropriately instead of using short names that seem useful at the time...

PaulS · Joined: 04 Jan 2008 Posts: 140

Nice breakdown.

I also didn't see this, since I run Symmetricom's NTP client on my Windows pyTivo server. I rarely get outside of +/- 1 second.
I believe that Windows only comes equipped with an SNTP server, which is good for roughly +/- 20 seconds, if I'm not mistaken.
Is that good enough ?

rdian06 · Joined: 12 Apr 2008 Posts: 362

A +20 second drift should be pretty inconsequential. It's the people who have 20 minutes to hours off (because of bad timezone settings) that would really be affected.

PaulS · Joined: 04 Jan 2008 Posts: 140

Then the w32time Time Service from Windows should be sufficient for most folks.
Mac and Linux folks can use ntpd, which should be integrated with most (if not all) distros.

Unless, of course, there's some means to integrate some form of NTP functionality directly into pyTivo.

ebf · Joined: 21 Mar 2008 Posts: 47

Interesting that the first time I have this starting-at-the-end of program issue, this thread has bubbled back up to the top of the Support forum! I used the "Save to VRC" trick to get the single program to to play from the beginning. I have for sometime found that many of my transfers start playing at one minute into the program. Today is the first time I have had a program try to play right at the end. I have reset my PC's clock and will see if that will fix the issue for me.

rdian06 · Joined: 12 Apr 2008 Posts: 362

welldressedmatt · Joined: 04 Sep 2008 Posts: 13

This seems to be happening to me - fully transferred recordings start exactly 10 minutes in, unless the video is less than 10 minutes, in which case it queues up at the end (with no possibility of rewinding).

I'm running pyTivo on Mac OS 10.4 Tiger with a Series 2 Tivo.

EDIT: My Mac was set 10 minutes fast, I fixed this by checking 'Set Date & Time Automatically' under Date & Time in System Preferences.

-M@