pyTivo Discussion Forum Forum Index pyTivo Discussion Forum
Answers and the development of pyTivo a TiVo transcoding server
 
 FAQFAQ   SearchSearch   MemberlistMemberlist   UsergroupsUsergroups   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 

gmd's python metadata generator
Goto page Previous  1, 2, 3, 4, 5, 6, 7, 8  Next
 
Post new topic   Reply to topic    pyTivo Discussion Forum Forum Index -> Other Apps
 View previous topic :: View next topic  
Author Message
sicklybutsexy



Joined: 05 Mar 2008
Posts: 91
Location: chicago

PostPosted: Wed Jun 04, 2008 11:41 am    Post subject: Reply with quote

Working again but I've noticed that for the last several months there are no descriptions given in the metadata files...Is anyone still using this program? I think it's great because it can be set up automatically. I guess I'll just use metadatagen and stop replying to myself...
Back to top
View user's profile Send private message
gmd



Joined: 08 Jan 2008
Posts: 28

PostPosted: Thu Jun 05, 2008 5:51 am    Post subject: Reply with quote

sicklybutsexy wrote:
Working again but I've noticed that for the last several months there are no descriptions given in the metadata files...Is anyone still using this program? I think it's great because it can be set up automatically. I guess I'll just use metadatagen and stop replying to myself...


Really? What are you generating metadata for? I've been using it for Lost/House mostly (my TV consumption is pretty limited now with a 9 month old Smile and those seem to work fine.

_________________
pyTivoMetaThis - metadata via python - http://trac.kurai.org
Back to top
View user's profile Send private message
sicklybutsexy



Joined: 05 Mar 2008
Posts: 91
Location: chicago

PostPosted: Thu Jun 05, 2008 5:08 pm    Post subject: Reply with quote

Sorry, still working fine for TV files but for "movie" files there hasn't been a description for a movie I have since April. I have obtained a lot of movies since then and no description. If I erase a metadata file for a movie it previously got a description for and then run the program then it also creates a metadata fie without a description. did something change with the IMDB? I can understand with a 9 month old (believe me, it only gets worse, or better depending on how you look @ it).
Back to top
View user's profile Send private message
stevetony



Joined: 05 Jun 2008
Posts: 16

PostPosted: Thu Jun 05, 2008 5:45 pm    Post subject: Reply with quote

sicklybutsexy wrote:
Sorry, still working fine for TV files but for "movie" files there hasn't been a description for a movie I have since April. I have obtained a lot of movies since then and no description. If I erase a metadata file for a movie it previously got a description for and then run the program then it also creates a metadata fie without a description. did something change with the IMDB? I can understand with a 9 month old (believe me, it only gets worse, or better depending on how you look @ it).


I just started using this today and seeing the same thing - no descriptions for movies. I was using MetaGenerator, but movie-xml.com has been down almost entirely for the past 4-5 days. Bummer.
Back to top
View user's profile Send private message
andrezero



Joined: 06 Jun 2008
Posts: 2

PostPosted: Sat Jun 07, 2008 7:04 pm    Post subject: Reply with quote

I've just started using this and it's really great!

I have found one problem that I keep having though. Some recordings generate the following error:

Code:

Traceback (most recent call last):
  File "C:\Program Files\pyTivo\pyTivoMetaThis.py", line 431, in <module>
    main()
  File "C:\Program Files\pyTivo\pyTivoMetaThis.py", line 421, in main
    (seriesInfoXML, seriesid) = getSeriesId(MirrorURL, series)
  File "C:\Program Files\pyTivo\pyTivoMetaThis.py", line 125, in getSeriesId
    print "Series Name: %s \t Series ID: %s\t Description: %s" % (e.findtext('Se
riesName'), e.findtext('id'), e.findtext('Overview'))
  File "C:\Python25\lib\encodings\cp437.py", line 12, in encode
    return codecs.charmap_encode(input,errors,encoding_map)
UnicodeEncodeError: 'charmap' codec can't encode character u'\u2013' in position


Am I correct in assuming that this is a problem with the character mapping and not with pyTivoMetaThis itself? Any idea how I could fix/work around this? I encountered this trying to get meta for episode 6 of Spectacular Spider-man. It did generate the first few fields of the meta but lacks the episode description...
Back to top
View user's profile Send private message
johnnyjet



Joined: 01 Jul 2008
Posts: 6

PostPosted: Fri Jul 25, 2008 8:59 am    Post subject: Getting a lot of 'invalid title: ""' errors Reply with quote

While trying to get metadata for some movies, I've started running into this error. I'm assuming it's some kind of formatting issue, since I know that these titles are in the imdb (after all, that's how I get the actual titles). I'm seeing the issue with well-known movies (Shrek 2, Dracula) as well as not-so-well-known titles (Forbidden Zone, Scooby-Doo and the Haunted Candy Factory).

ja
Back to top
View user's profile Send private message
andrezero



Joined: 06 Jun 2008
Posts: 2

PostPosted: Fri Aug 08, 2008 3:50 pm    Post subject: Reply with quote

I managed to "fix" my charmap problem. Apparently the problem was were the series info was printed (lines 124-126). I think the unicode character isn't being output to the screen correctly. I commented out the loop that prints the series info and that fixed the problem. The downside is if a show has multiple hits for a series, I can't see what they are!

I wish I could fix the mapping itself but I know just enough python to be dangerous!
Back to top
View user's profile Send private message
imatt



Joined: 21 Aug 2008
Posts: 4

PostPosted: Sun Sep 28, 2008 5:31 pm    Post subject: Reply with quote

I had the same problem with the metadata files not being written... but then I figured out what was going on.
It didn't play entirely friendly with my network shares.

Using:
PyTivo: wgw fork, Sept 25, 2008.
pyTivoMeta 0.19


Setup:
- pytivo and pytivometa run on a Vista64 box.
- files are both local and on a network share.

Pytivometa works fine when reading/writing to the local folder, but when accessing the network shares or files located in other directories, it will write the SERIESID file but not the individual metadata text file. This is repeatable.


I made a minor change and fixed this problem.
For running the code on WINDOWS:

Find:

Code:

        if (len(metadataText) > 1):
            outFile = open(f, 'w')
            outFile.writelines(metadataText)
            outFile.close()


Change to:

Code:

        if (len(metadataText) > 1):
            fpath = options.filedir + '\\\\' + f
            outFile = open(fpath, 'w')
            outFile.writelines(metadataText)
            outFile.close()



Make sure the indenting stays the same - ie, make sure the code lines up on the left like it did originally.

Basically all I did was tell it to put the full pathname in the value so the script would be able to place the metadata file. This was included in the SERIESID function, but not the metadata function.

I did not test this on the Movies function, but would probably need to be changed there as well.



There also appears to be an issue with the .meta folder only being created inside the folder that pytivometathis is being run from.


http://trac.kurai.org/trac.cgi/ticket/1
Ticket on this issue.
Back to top
View user's profile Send private message
rdian06



Joined: 12 Apr 2008
Posts: 463

PostPosted: Fri Oct 03, 2008 5:43 am    Post subject: Reply with quote

I really liked the idea of this script, but as many of you ran into, it has some rough edges.

So I looked through the code and fixed a few things. I'm posting my modified version here in case anyone else finds it useful.

Fixes:

* Added code to translate fancy double and single quotes (that can't be displayed on the Tivo) to standard double and single quotes

* Added mkv files to the list of files that are processed

* pyTivo metadata files allow UTF8 text encoding so that letters with various accents can be displayed on the Tivo, but the script dealt with UTF8 incorrectly by converting it to the shell's output encoding (or not at all causing the script to crash when printing to screen). I went through and corrected/added all the conversions to preserve UTF8 when outputting to the metadata file and convert properly for screen display (no more crashes due to unprintable characters). (Also means that if an actor's name has e or i with an accent, the e or i isn't just thrown away like it is when using MetaGenerator. I'm anal and used to correct these by hand when using MetaGenerator.)

* Fixed the problem that imatt described where when the -p <path> is used, metadata file are still written to the local directory from which the script was run instead of the directory containing the video file. My fix should be OS independent and works for both Movie and TV mode, however I don't understand how the -t option is supposed to work so I left the .meta directory path parts alone (If someone wants to explain to me how -t mode should work, I'd be happy to fix it too.)

* When prompting to identify a TV show with multiple possible matches, only displays the first three lines of the TV show description so the screen doesn't get flooded. Also made some formatting changes for readability. Did I mention the script no longer crashes with unprintable characters for this Smile

* When run in -m Movie mode, ignore processing files with the TV show format specifier (i.e. contain SxxExx in their filename) because they were being treated as movies causing spurious matches and metadata to be written

* Fixed the metadata output filenames being converted to all lower case. They now should match the case of the video file properly. (Again, I'm anal.)

* Rewrote the Movie metadata output code to produce output in the same order as Metagenerator. Should now properly output description when it is available (before the description only printed when there was an MPAA Rating). Displays IMDB rating at the end of the description like MetaGenerator. Calculates Star rating based on the IMDB rating like MetaGenerator. Suppresses including the whole production staff as vActors. Now only includes actors as vActors in IMDB credits order. Suppresses the same person from being displayed multiple times under vActor, vWriter and vDirector. (This primarily happened with vWriter, but if someone was a director and an actor on the movie, then they properly appear in both sections. You just won't get duplicates in the same section.)

* The MPAA ratings in Movie mode code to Tivo ratings conversion is a little weak, but it works for things that actually have the rating in IMDB.

* By default, the title field of the Movie metadata only contains the movie title. If you want the year to also be displayed at the end of the title, pass the -a option to the script

* Rewrote the TV metadata output code to produce output in the same order as Metagenerator. This makes it easier to eyeball the file because the seriesTitle, title, and description show up at the top (instead of at the bottom after all the actors as originally). Added mappings for the vProgramGenre and vSeriesGenre info.

* By default the seriesTitle and title (aka the episode title) fields show the literal series title and episode title. This works okay when the seriesId is available in your Tivo's guide data and grouping kicks in, but if the seriesId isn't in your guide (because you've ripped a bunch of old shows from DVD) this is annoying. Passing -a (alternative output) to the script will add more information to seriesTitle and title as explained below.

When grouping kicks in, the seriesTitle field is ignored for naming the group in the Now Playing list. The Tivo takes the series name from it's guide data. When you navigate into the group, the video's appear to be listed in transfer order (newest transfer first, but I haven't done enough testing to be certain) and their titles (aka episode titles) are displayed. If you then select one of the episodes, the actual seriesTitle field contents are displayed at the top with the title (aka episode title) in quotes with year and description following. Passing -a to the script, the title (aka episode title) becomes "S<XX>E<YY> <EpisodeTitle>" where XX is the season number and YY is the episode number so that when grouping kicks in, you can easier discern the intended order of the episodes.

If grouping doesn't kick in (i.e. no seriesId in your guide data), files are listed individual in your Now Playing List (newest transfer on top if stored by date) and the seriesTitle is displayed. Using the standard output, this is very confusing as each video has the same seriesTitle. If you pass -a to the script, seriesTitle becomes "<SeriesName> S<XX>E<YY> <EpisodeTitle>" where <XX> and <YY> are the season and episode numbers. At least this way you can tell more about the video without clicking through.



pyTivoMetaThis-rdian06-0.19c.zip
 Description:

Download
 Filename:  pyTivoMetaThis-rdian06-0.19c.zip
 Filesize:  6.78 KB
 Downloaded:  35 Time(s)



Last edited by rdian06 on Fri Oct 03, 2008 11:22 pm; edited 2 times in total
Back to top
View user's profile Send private message
danimal4326



Joined: 14 Apr 2008
Posts: 24

PostPosted: Fri Oct 03, 2008 1:41 pm    Post subject: Reply with quote

Code:

xxxx@timmy /media/video/TV/Californication/Season 2 > ll
total 239024
-rwxr--r-- 1 xxxx users 244753846 2008-09-29 00:00 Californication.S02E01.HDTV.XviD-NoTV.[VTV].avi*

xxxx@timmy /media/video/TV/Californication/Season 2 > pyTivoMetaThis.py -d

Console Input encoding: UTF-8
Console Output encoding: UTF-8
Metadata File Output encoding: utf8

fileList before cull: ['Californication.S02E01.HDTV.XviD-NoTV.[VTV].avi']
fileList after cull: ['Californication.S02E01.HDTV.XviD-NoTV.[VTV].avi']

--->working on: Californication.S02E01.HDTV.XviD-NoTV.[VTV].avi

Series: Californication
Season: 2 Episode: 1
Looking for .seriesID file in ./Californication.seriesID
Searching for series with name Californication
seriesXML: Using URL http://thetvdb.com/api/GetSeries.php?seriesname=Californication
Creating ./Californication.seriesID
getSeriesInfoXML: Using URL http://thetvdb.com/api/0403764A0DA51955/series/80349/en.xml
getEpisodeInfoXML: Using URL http://thetvdb.com/api/0403764A0DA51955/series/80349/default/2/1/en.xml
Returning episodeInfoXML:
<Element Data at 29d6cb0>
In formatEpisodeData

Working on seriesTitle
seriesTitle : Californication
Completed -> seriesTitle : Californication

Working on title
title : Slip Of The Tongue
Completed -> title : Slip Of The Tongue

Working on originalAirDate
originalAirDate : 2008-09-28
Completed -> originalAirDate : 2008-09-28T00:00:00Z

Working on description
Traceback (most recent call last):
  File "/home/danc/bin/pyTivoMetaThis.py", line 586, in <module>
    main()
  File "/home/danc/bin/pyTivoMetaThis.py", line 583, in main
    formatEpisodeData(episodeInfo, metadataFileName)
  File "/home/danc/bin/pyTivoMetaThis.py", line 264, in formatEpisodeData
    if (options.debug): print "%s : %s" % (tvTag.encode(out_encoding, 'replace'), text.encode(out_encoding, 'replace'))
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 172: ordinal not in range(128)

xxxx@timmy /media/video/TV/Californication/Season 2 > ll
total 239028
-rwxr--r-- 1 xxxx users 244753846 2008-09-29 00:00 Californication.S02E01.HDTV.XviD-NoTV.[VTV].avi*
-rw-r--r-- 1 xxxx users         5 2008-10-03 16:39 Californication.seriesID

Back to top
View user's profile Send private message
rdian06



Joined: 12 Apr 2008
Posts: 463

PostPosted: Fri Oct 03, 2008 5:37 pm    Post subject: Reply with quote

Argh, random smart quote char in the description field.

Ironically, if you leave off the debug flag things work. It's the debug print to console that is actually triggering the error.

Back to the drawing board.
Back to top
View user's profile Send private message
rdian06



Joined: 12 Apr 2008
Posts: 463

PostPosted: Fri Oct 03, 2008 8:01 pm    Post subject: Reply with quote

I fixed the problem danimal4326 pointed out. Turns out I was encoding Unicode strings to the UTF-8 representation correctly, but for certain debug output, I was doing a double encode which is caused the blow up.

I scanned through the script again and think I fixed all possible places I was double encoding (or needlessly encoding in the first place.)

Uploaded a new version. Check the previous post for pyTivoMetaThis-rdian06-0.19b.zip.

As for that Californication episode S02E01's description. It contains a fancy single quote character rather than a simple apostrophe. I logged into thetvdb.com and fixed it since there really is no reason to use that special Unicode character. And once the fancy single quote character is transferred to the Tivo, it comes up as an empty box because the Tivo doesn't have a corresponding font character for it.
Back to top
View user's profile Send private message
rdian06



Joined: 12 Apr 2008
Posts: 463

PostPosted: Fri Oct 03, 2008 11:24 pm    Post subject: Reply with quote

So it seems quite a few recent shows have the fancy quotes instead of standard ones.

So I added some code to translate fancy quotes to standard ones. Updated to pyTivoMetaThis-rdian06-0.19c.zip
Back to top
View user's profile Send private message
wmcbrine



Joined: 04 Jan 2008
Posts: 514

PostPosted: Sat Oct 04, 2008 10:45 am    Post subject: Reply with quote

I wish TiVo would beef up their character set. You can send it utf-8, but really all it knows are 256 characters, IIRC.
_________________
My pyTivo fork
Back to top
View user's profile Send private message
danimal4326



Joined: 14 Apr 2008
Posts: 24

PostPosted: Tue Oct 07, 2008 5:39 am    Post subject: Reply with quote

rdian06 wrote:
I fixed the problem danimal4326 pointed out. Turns out I was encoding Unicode strings to the UTF-8 representation correctly, but for certain debug output, I was doing a double encode which is caused the blow up.

I scanned through the script again and think I fixed all possible places I was double encoding (or needlessly encoding in the first place.)

Uploaded a new version. Check the previous post for pyTivoMetaThis-rdian06-0.19b.zip.

As for that Californication episode S02E01's description. It contains a fancy single quote character rather than a simple apostrophe. I logged into thetvdb.com and fixed it since there really is no reason to use that special Unicode character. And once the fancy single quote character is transferred to the Tivo, it comes up as an empty box because the Tivo doesn't have a corresponding font character for it.

Thanks, that fix it... for TV shows.. seems I get the same error for Movies...
Back to top
View user's profile Send private message
Display posts from previous:   
Post new topic   Reply to topic    pyTivo Discussion Forum Forum Index -> Other Apps All times are GMT - 8 Hours
Goto page Previous  1, 2, 3, 4, 5, 6, 7, 8  Next
Page 5 of 8

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You can download files in this forum
Site is in NO WAY affiliated with TiVo Inc

Powered by phpBB © 2001, 2005 phpBB Group
phpBB SEO
[ Time: 0.2911s ][ Queries: 14 (0.1614s) ][ Debug on ]