pyTivo Discussion Forum Forum Index pyTivo Discussion Forum
Answers and the development of pyTivo a TiVo transcoding server
 
 FAQFAQ   SearchSearch   MemberlistMemberlist   UsergroupsUsergroups   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 

Transcoding Speed
Goto page 1, 2  Next
 
Post new topic   Reply to topic    pyTivo Discussion Forum Forum Index -> Support
 View previous topic :: View next topic  
Author Message
Ganlron



Joined: 27 Mar 2008
Posts: 5

PostPosted: Thu Mar 27, 2008 6:20 am    Post subject: Transcoding Speed Reply with quote

I'm running on pyTivo on my Macbook Pro and while it's working awesome, I've been avoiding transcoding by using VisualHub to convert the video files into mpgs as when I attempt to transcode on the fly (say from .avi or .mp4), it takes a large amount of time.

Just curious if there's any tweaks I can make to ffmpeg to have it transcode faster? It just seems odd that VisualHub which uses ffmpeg as well, can convert the file in 30 minutes where as when I use it via pyTivo in 30 minutes I've transcoded all of 10 minutes of the video.
Back to top
View user's profile Send private message
krkeegan
Site Admin


Joined: 04 Jan 2008
Posts: 412
Location: Los Angeles, CA

PostPosted: Thu Mar 27, 2008 12:24 pm    Post subject: Reply with quote

You know another user is reporting slow transcoding speeds on a mac too. I am beginning to think there is some issue in there. Can you check why the python CPU usage is while pyTivo is transcoding? It should be only 1% while ffmpeg should be 70-90%.

Other Thread
Back to top
View user's profile Send private message Visit poster's website
Iluvatar



Joined: 29 Feb 2008
Posts: 13

PostPosted: Thu Mar 27, 2008 12:56 pm    Post subject: Reply with quote

using pytivo on my mac for the past 6 months i can say it was a definite problem. ffmpeg very rarely went over 40% while python process was the rest. always took a very long time to transcode on the mac and finally i had to go back to using the pc laptop. was tried with 10.4.11 and up to 10.5.1 with the latest python packages and various ffmpeg versions up to the latest available. i updated macports and various dependencies for ffmpeg on a weekly basis.

python on the mac is very slow. Sad
Back to top
View user's profile Send private message
Ganlron



Joined: 27 Mar 2008
Posts: 5

PostPosted: Thu Mar 27, 2008 12:58 pm    Post subject: Reply with quote

Actually you know, I saw the exact same thing. The pyTivo process varied in CPU utilization between 1% to 90% while ffmpeg was hovering constantly at about 17%.

I'll test again tonight when I'm home and post some ps captures

Also came across an odd issue when trying out the hack83 folder browsing and multiple shares. If I had a share with folders it worked without issue, however if the share had no folders, only videos when I viewed the information on the tivo about a video and went back the share showed as empty. I had to return to the now playing list and then back to the share to choose another file. There was an error from pyTivo when I checked it, I'll see if I can replicate it again tonight and post the error.
Back to top
View user's profile Send private message
krkeegan
Site Admin


Joined: 04 Jan 2008
Posts: 412
Location: Los Angeles, CA

PostPosted: Thu Mar 27, 2008 2:01 pm    Post subject: Reply with quote

Your error with hack83 sounds possible. I am not sure I ever tested that situation. I will look into that.

If you want to try a beta concept that wgw is working on try this:

Edit plugins/video/transcode.py


Find the following line: (Around line 57)
Code:
     ffmpeg = subprocess.Popen(cmd, stdout=subprocess.PIPE)


Replace with:
Code:
     ffmpeg = subprocess.Popen(cmd, bufsize=512*1024, stdout=subprocess.PIPE)


A user back in november reported that lowered python cpu usage
Back to top
View user's profile Send private message Visit poster's website
Ganlron



Joined: 27 Mar 2008
Posts: 5

PostPosted: Thu Mar 27, 2008 2:08 pm    Post subject: Reply with quote

Okay, I'll give that a whirl. I just did a transcoding of a small video to get a ps and pyTivo consistently gobbles up processor usage. Here's some examples:

derek 140 80.4 0.9 100868 19232 ?? R 5:56pm 0:18.08 /System/Library/Frameworks/Python.framework/Versions/2.5/Resources/Python.app/Contents/MacOS/Python /Applications/pyTivo/pyTivo.py
derek 338 43.2 0.2 614064 5220 ?? S 6:02pm 0:07.81 /opt/local/bin/ffmpeg -i /Volumes/The Library/TiVoShows/Movies/daisy_crazy.avi -vcodec mpeg2video -r 29.97 -b 4096K -maxrate 17408k -bufsize 1024k -aspect 4:3 -s 544x480 -comment pyTivo.py -ab 384k -ar 48000 -acodec ac3 -f vob -


derek 140 83.3 0.9 100868 19232 ?? S 5:56pm 0:35.63 /System/Library/Frameworks/Python.framework/Versions/2.5/Resources/Python.app/Contents/MacOS/Python /Applications/pyTivo/pyTivo.py
derek 338 37.2 0.2 614032 5188 ?? R 6:02pm 0:17.10 /opt/local/bin/ffmpeg -i /Volumes/The Library/TiVoShows/Movies/daisy_crazy.avi -vcodec mpeg2video -r 29.97 -b 4096K -maxrate 17408k -bufsize 1024k -aspect 4:3 -s 544x480 -comment pyTivo.py -ab 384k -ar 48000 -acodec ac3 -f vob -

derek 140 73.3 0.9 100868 19236 ?? R 5:56pm 0:44.74 /System/Library/Frameworks/Python.framework/Versions/2.5/Resources/Python.app/Contents/MacOS/Python /Applications/pyTivo/pyTivo.py
derek 338 46.9 0.2 614040 5200 ?? R 6:02pm 0:22.01 /opt/local/bin/ffmpeg -i /Volumes/The Library/TiVoShows/Movies/daisy_crazy.avi -vcodec mpeg2video -r 29.97 -b 4096K -maxrate 17408k -bufsize 1024k -aspect 4:3 -s 544x480 -comment pyTivo.py -ab 384k -ar 48000 -acodec ac3 -f vob -


I'll see if that beta concept makes a difference.
Back to top
View user's profile Send private message
sdwr98



Joined: 25 Mar 2008
Posts: 4

PostPosted: Thu Mar 27, 2008 3:09 pm    Post subject: Reply with quote

Well, that most definitely chopped off the Python CPU utilization for me - it's now hovering around 1.1%

ffmpeg is still only going at about 17-20%, but I am going to put the -threads 2 option on and see if that helps.

Scott
Back to top
View user's profile Send private message
Ganlron



Joined: 27 Mar 2008
Posts: 5

PostPosted: Thu Mar 27, 2008 3:40 pm    Post subject: Reply with quote

Yup, huge difference here too with that change.

derek 9879 98.9 0.3 614152 5328 s000 R+ 7:37pm 0:26.91 /opt/local/bin/ffmpeg -i /Volumes/The Library/TiVoShows/Movies/daisy_crazy.avi -vcodec mpeg2video -r 29.97 -b 4096K -maxrate 17408k -bufsize 1024k -aspect 4:3 -s 544x480 -comment pyTivo.py -ab 384k -ar 48000 -acodec ac3 -f vob -
derek 1948 1.2 0.9 101972 19720 s000 S+ 7:20pm 0:04.03 /System/Library/Frameworks/Python.framework/Versions/2.5/Resources/Python.app/Contents/MacOS/Python /Applications/pyTivo/pyTivo.py


Let me know if -threads 2 made much of a difference. I noticed when VisualHub does the transcoding it uses -threads 4 so might
Back to top
View user's profile Send private message
krkeegan
Site Admin


Joined: 04 Jan 2008
Posts: 412
Location: Los Angeles, CA

PostPosted: Thu Mar 27, 2008 5:19 pm    Post subject: Reply with quote

very cool, glad to hear that at least helped part of the problem.

Keep me posted about the threads 2 idea.
Back to top
View user's profile Send private message Visit poster's website
real_armooo
pyTivo Creator


Joined: 23 Mar 2008
Posts: 42

PostPosted: Thu Mar 27, 2008 6:43 pm    Post subject: Reply with quote

sdwr98 wrote:
Well, that most definitely chopped off the Python CPU utilization for me - it's now hovering around 1.1%

ffmpeg is still only going at about 17-20%, but I am going to put the -threads 2 option on and see if that helps.

Scott


If you are transcoding faster than the tivo is downloading, the ffmpeg output buffer will fill up and the write calls will block limiting the cpu usage of ffmpeg.
Back to top
View user's profile Send private message
krkeegan
Site Admin


Joined: 04 Jan 2008
Posts: 412
Location: Los Angeles, CA

PostPosted: Thu Mar 27, 2008 7:47 pm    Post subject: Reply with quote

Very true. This may be a bandwidth limitation too if you are not using a wired connection.
Back to top
View user's profile Send private message Visit poster's website
Ganlron



Joined: 27 Mar 2008
Posts: 5

PostPosted: Fri Mar 28, 2008 3:29 am    Post subject: Reply with quote

Well I don't know if that is the issue. I'm using a wireless connection on the TiVo but in comparison a 1GB pre-transcoded file finishes transferring to the TiVo in less than 10 minutes.

Now with the beta change, transcoding on the fly is at least watchable. I put a 700MB MP4 file up on the share and it's streaming to the TiVo in a watchable fashion, though the transcoding/transfer appears to just be keeping ahead of the TiVo playing the file. I may see if I can try the -threads 2 myself tonight and see if the file finishes transcoding quicker (VisualHub which I normally use to pre-transcode the files seems to use -threads 4 and it'll finish a normal file in 20-30 minutes)
Back to top
View user's profile Send private message
krkeegan
Site Admin


Joined: 04 Jan 2008
Posts: 412
Location: Los Angeles, CA

PostPosted: Fri Mar 28, 2008 8:01 pm    Post subject: Reply with quote

Ganlron wrote:
Also came across an odd issue when trying out the hack83 folder browsing and multiple shares. If I had a share with folders it worked without issue, however if the share had no folders, only videos when I viewed the information on the tivo about a video and went back the share showed as empty. I had to return to the now playing list and then back to the share to choose another file.


Soo, I looked into this. I don't have the same issue. I made a new share that had no subfolders in it. I browsed viewing the details of the files and exiting and everything was still in the list when I exited back. I tested this with both linux and windows machines as the server. I am using TiVoHD so possibly this is an S2 issue that I can't test? If others can reproduce this it might be helpful in diagnosing. But until them I am going to close this issue.

I will however note that I saw some very very odd behavior from my TiVoHD a few times, but I am no longer able to replicate it. At one point while exiting from viewing file details I breifly saw old school black and white static on my LCD TV. For a while after this it seemed that TiVoHD was alternating resolutions each time when looked at the file descriptions. This has since gone away and I can no longer repeat it. At the moment I am chalking this up to some odd TiVoHD bug.
Back to top
View user's profile Send private message Visit poster's website
dizziness



Joined: 22 Jan 2008
Posts: 34

PostPosted: Mon Apr 07, 2008 4:07 pm    Post subject: Reply with quote

Recent WGW stem fixed the Python CPU usage problem on the Mac. Python now runs at %0.8 and FFMPEG over 100%. Excellent work guys!
Back to top
View user's profile Send private message Visit poster's website AIM Address MSN Messenger
krkeegan
Site Admin


Joined: 04 Jan 2008
Posts: 412
Location: Los Angeles, CA

PostPosted: Mon Apr 07, 2008 6:05 pm    Post subject: Reply with quote

cool, and I have not seen any adverse effects on windows or linux so I think we are good to include this.
Back to top
View user's profile Send private message Visit poster's website
Display posts from previous:   
Post new topic   Reply to topic    pyTivo Discussion Forum Forum Index -> Support All times are GMT - 8 Hours
Goto page 1, 2  Next
Page 1 of 2

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You can download files in this forum
Site is in NO WAY affiliated with TiVo Inc

Powered by phpBB © 2001, 2005 phpBB Group
phpBB SEO
[ Time: 0.2458s ][ Queries: 13 (0.1371s) ][ Debug on ]