Hi everyone,

I made this (hopefully simple) guide on how to IVTC (inverse telecine) your videos before encoding. Since most videos (not all, but a lot of them) are filmed at 24 frames per second, you don't want to encode the 30 fps material that you get from PO, ScreenPlay, etc. Unfortunately, pretty much everyone seems to be doing it wrong, and I am hoping that will change. The only person/service I've seen actually doing this correctly so far is SmashVidz.

Anyway, this does require Windows, but this process is going to give you better results than anything else, I promise. Better encodes, smoother frame rates, and more efficient playback in Video SL.

Here's the link: www.youtube.com view
Be sure to watch in 720p as you'll be able to read the text more easily.

Note: I left out a few things (such as replacing audio with CD-source stuff) just for simplicity; I am just focusing on the video in this tutorial.

Please let me know if there are any questions and I'd be more than happy to help. I really hope more people can start using this method, it'll be better for all of us.

Brett

Thanks for updating the guide Brett. I've been getting nice results with your old guide.

Thank you!

Dope man...

thanks for the video! It will help a lot of users here!

very nice video!!!! thank you

Do you have any suggestions for rendering preferences in Sony Vegas after quick edit of a video?

Well, when I edit in Vegas, I always do something similar to this process first, except instead of encoding with MeGUI, I create an uncompressed AVI with VirtualDub (just load the .avs into VirtualDub and then save the uncompressed AVI).

Then, I edit with that file in Vegas, and then render, again, an uncompressed AVI out of Vegas. After that, I create an .avs script that looks like this:

DirectShowSource("vegasproject.avi")
ConvertToYV12()

Then load that .avs into MeGUI and encode as normal. This way, you're only going through one encoding stage.

Hope that helps!

Also, it is ESSENTIAL that your Vegas project settings are equal to that of the AVI you're editing with. So, for instance, if you've got a 640x360 video at 23.976 fps, make sure you set that in Vegas as well.

Cool. I will definately try this out.

"a little bit of a learning curve?"

haaaaaaaaaa!

great post man.

maybe u guys can explain this one to me.
I understand how when u go from a 24 to 29.97/30 fps it does that by making
dupes (like u mentioned every 5th frame)
but you also said to keep it at the original fps rate (so, why is there a dupe frame?)

thanks man, so informative

It's just because the DVD spec requires 29.97 fps. So, that's just the method that's employed to handle 23.976 fps material -- three progressive frames, and two combed frames. Fortunately, Blu-ray allows real 23.976 fps progressive stuff, but I doubt we'll be seeing any music videos on Blu any time soon.

Oh, ok.
so the original media is 23.976 but when its put onto the DVD it shoots to 29.97?
then we are just down-converting it back to its original format of 23.976

if they all come up saying 29.97, how do we know it was initially 23.976 and requires
the duplicate frames removed?
and will VSL have any issues vith a file rendered at anything other than 29.97
I thought I read that 29.97 is the preffered FPS rate?


sorry for the Inquisition Brett... Im at that point where I know just enough to be dangerous (to myself) in the realm of videos

Nah, it's all good! I'm glad people are asking questions.

The easiest way to find out whether to IVTC it is to put the TDeint() line into the AviSynth script (and don't do the Decimate line yet). Then, load into MeGUI and start hitting the right arrow on your keyboard. If you see four frames and then a duplicate, then you know to remove that. If you keep seeing constant frames already, then you've got 29.97 material.

23.976 material actually runs better in VSL than 29.97. Less frames to decode = less CPU required. It also means that when the software has to blend frames (which it pretty much always does since you're adjusting pitch), it's blending from the raw individual frames, and not an already blended source (which is what most crap deinterlacers do.. they just blend the two combed frames). Just gives you better performance all around.

VSL actually supports quite a bit, as long as it's encoded properly. Heck, I even have some 59.94fps files in my library that work just fine.

Brett, thanks for the tutorial. I alreadly went through it and have got to the point of trying some encodes using your workflow. However, I have lot of VOB's that i ripped from PAL DVD's which themselves are at 25fps. With these type of sources is necessary to remove the extra (duplicate frames). If does not not seem to make an difference for me.

Ah, that one is tricky. If the video was actually recorded in PAL (meaning it's actual 25fps source) then you don't need to do anything.. just deinterlace and encode.

However, if the video was originally film or NTSC (23.976 or 29.97) and then converted to PAL, you're pretty much out of luck, as they have to apply frame blending to those sources to get them to look good for PAL uses. If you can get your hands on other sources for the video, definitely do so.

Cool, it is actually the original PAL and actual 25fps. The workflow has already helped with one of two other files though. I do have PO and Screen Play DVD's so either way very very useful. Thanks again.

Can those settings be achieved using Sorenson Squeeze. I've been using Phat Al's encoding settings found here. How does that compare?
www.serato.com

I'm not sure, as I don't have experience with Sorenson Squeeze. I did some searching and I don't think that Squeeze has the ability to perform IVTC. It can, however, crop and output as widescreen if appropriate (you'd have to do this manually, though).

Of course Squeeze has the ability to do IVTC, it's a professional encoder.

Look under the filters section

I stand corrected.

thanks DJ Brett for the guide, and for your help.


thanks Justin, I will take a look at the IVTC settings in Sorenson.

Sorenson Squeeze is picky with IVTC. I remember I was trying to use it on an older version and my files would become out of sync when using the IVTC filter.

nik39 and I tried convincing everybody that the pulldown was ruining their videos years ago but no one seemed to care. they were happy to deinterlace the pulled down material (ruining it even further).

i'm glad to see some of you finally taking notice that the you need to remove the pulldown before editing or converting to mp4.

i just wish all dvd material was the same so the same process could be used for every video.

Oh btw, I'd bet that smashvidz gets a lot of their videos in the native frame rate, before the pulldown was added.

It's not that we didn't care. This is just the first tutorial that got posted.

thanks d-twizzle, so you're currently not using IVTC setting on Sorenson. I wonder how the new version can handle it.

before my last tutorial i did, i actually was going to post a tutorial on pulldown removal. the topic was pretty much ignored by everyone so i just moved on to something different.

i'm just glad people are starting to take notice. the absolute worst are the sites that happily ruin pulled down material by deinterlacing it, and then selling the videos like that. this is the worst because once they've been deinterlaced, you'll never get back the 23.98 frames in tact.

i have plenty of ruined videos from the early days so i know what i'm talking about.

maybe a tutorial for mac users is still in order though. compressor, after effects, custom scripts and shake can also remove pulldown on the mac. after effects is the only one though that can auto-detect the cadence AND allow you to override the detected cadence and supply your own pattern. compressor just auto-detects and you pretty much need to supply the cadence to shake in order for it to work.

also, i didn't watch his tutorial but I'd be concerned about source material that was shot with the so called "advanced pulldown" or 2:3:3:2 cadence. Obviously if the pulldown removal only knows how to deal with 3:2 (or 2:3, whichever you prefer), then it will seriously bungle source frames with 2:3:3:2 pulldown.

Anyway, glad to see people doing this. Now get someone from all those crap sites ruining videos to read this thread.

oh I forgot to mention that AE can remove 3:2 and 2:3:3:2

Is there any limitations on the operating system to do this? Would someone need XP or above?

Are you on Windows 98? JK

if you do this that would be brilliant..... the mac tutorial i mean.... im trying to ween myself off PC!

Yeah, I'd like to know how to do this on the Mac as well. I still use Windows for my primary workstation OS but it would be nice to know.

A-Swift, there are ways for AviSynth to handle 2:3:3:2 pulldown material, however I've _never_ seen this used in any of the footage I edit.

Yeah, I've been trying to get people on this for two years now but, indeed, nobody seems to care (or understand the difference). Quite frustrating...

Nice tutorial Brett, thanks. Now I gotta install bootcamp...

I'll send you some. hahhaa.

we just shot a music video using HVX200. these are VERY popular cameras for shooting music videos.

Do all videos then have the pulldown? I have started to use this process, and I am taking each video one on one, and adjusting the crop size, but I want to make sure I am Dec(5) when I should be, and leaving it along when I should.

Also, rendering MPEG4's this way, how does that affect the keyframe? I was always understanding that Serato used keyframes as reference points in the song..

While there are tools on the Mac that can do IVTC, both commercial (Compressor, Cinema Tools) and freeware (JES Deinterlacer), none of them do even close to a good job of dealing with broken cadence material, such as most music videos shot on film, telecined to standard def, and then edited in a 60i environment.

As far as advanced pulldown, I have NEVER seen it used in final delivery. Advanced pulldown is usually added by 24p capable cameras to facilitate easy removal during ingestion into an NLE. The project could then be edited naively at 24p and a standard, 3:2 pulldown added when the final edited master is output to tape.

AviSynth can handle this. Granted, there are a few frames (lasting less than a second) each time the cadence is broken where it can get a little choppy, but it quickly figures out the cadence again and picks up correctly.

As I mentioned in the video, most require IVTC, but some do not. Easiest way to figure it out is to start with just TDeint() in the script and not the Decimate(5). Then in the preview window, start seeking frame-by-frame. If you see four frames and then a duplicate, four frames then a duplicate, four frames then a duplicate, etc. then that means you need to add the Decimate(5) line. If not, that means it's a 30fps/60i source, so just deinterlace with TDeint() and you're good.

No, not all videos have pulldown.

True

Keyframes are not related to pulldown at all.

If you are rendering a mp4 from a source with 3:2 pulldown with keyframes every 15 frames, there will a keyframe every 15 frames in the mp4

If you are rendering a mp4 from a source without 3:2 pulldown with keyframes every 15 frames, there will a keyframe every 15 frames in the mp4

Right on a-swift, but I do not see anywhere in the MeGui interface where I can dictate a keyframe every certain frames...

Thanks for the quick response as well DJ Brett B

thank you for sharing this wealth of knowledge!

is there a way to batch ivtc a group of vob files with your method?

especially if one were to organize the vobs so that they were seperated by whether they were originally shot in 24p or 29.97 etc.

also for us mac users, has anybody been had this level of precision using ivtc within compressor? thanks in advance...

on the mac, compressor will auto detect the cadence when doing reverse telecine. it's very easy.

after effects will let you autodetect the cadence, and override the detected setting if you like.

Check the "Advanced settings" box when you're configuring the x264 settings and add "--keyint 15" (or whatever you want--15 means every 15 frames). I've _never_ messed with that setting, just let x264 do its thing, and I've always had perfect performance. But that's just me.

I've been trying to figure out a solution to this. Problem is, I'm such a nerd that I really tweak each video individually. And replacing audio will always be a manual thing, so I've never looked into the idea too much. However, I am 100% sure it's possible if you're willing to write something to do it... I'll leave that to someone else though :P

you can batch vobs that have different cadence?

i don't know. can you?

the vob's could be organized into groups with the same cadence/parameters... and then batched that way...

basically this question is for those of us who are getting ready to redo our libraries again with this superior method and hope to avoid doing every vob file individually.

No, they'd have to have the same cadence.

I saw another web tutorial on this and they suggested making 5 folders, 1 for each cadence and then sorting your videos into them. Then you could batch all videos at the same time. Each folder at a time.

Could you possibly find the link to that? I'm very curious about how they achieved that.

it was a tutorial for doing reverse telecine with mencoder (command line). i think it was linked off the doom9 forums. i'll find it.

Awesome. Finally an interesting thread in here :) Props for all the information and the nice tutorial, Brett B.

I have a few questions about the whole process (most of these questions are not IVTC related per se), hopefully someone will be able to answer them.

1. Scaling. The common music video from a DVD will be at 720x480 with a PAR 0f 0.889 and a DAR of 4/3, which results finally in the given 640x480 (non-widescreen). Now, why are we converting the videos which are at 720 to 640? Wouldn't it make sense to try to avoid any additional rescaling (any real rescaling is like what for audio files means resampling which you always try to avoid)? The funny thing is... my tool does it also, so it some point it looks like I also agreed that this is the best approach - but I can't remember the answer :-(

2. Cropping. (This is somehow related to question 1) In your script you are cropping first, then resizing to 640x480. If you crop before you resize, then this means you are actually again resizing with an most likely not even factor, which also means you are degrading the picture. Wouldn't it be better to crop as the final step? (Okay, then VSL has to resize them later, but since VSL does resizing anyway, let VSL do it all in one step, saving as much quality as possible by avoiding multiple rescalings)

3. Decimate.

3.a Now when you go from 24fps to 29.97fps you have to "duplicate" a frame otherwise things will not match up. What I never understood is the timing. At 24fps you would have a time distance of 1/24s between each frame, with 29.97fps it is obviously 1/29.97s. Now if you spread 4 frames to 5 frames, doesn't it mean the you will introduce some judder? This is the first part of the question. Now comes the second part

3.b When you do the IVTC process and the decimate process, This means basically that you have to readjust the timing, since... well, forget it. This is not a question. Haha :) But this means that during IVTC you *must* use decimate otherwise you will end up with judder as well and will also mess up the timing since you have 29.97 frames (or is it 30?) but since you specified a target fps-rate of 24 the video will run at a different speed.

3.c Do you know if any tool which is able to show for each frame the exact timeposition? This is important for me with VFR, since handbrake seems to rely on VFR.

4. Cadence. Is anyone able to give an estimation about how many music videos have a fixed cadence, and how many have a broken cadence where you either need to break the scenes into parts and IVTC them seperately or alternatively, use an adaptive IVTC algorithm?


And please, if anyone finds some terms which I used in a wrong way (I am still a noob when it comes to video) please, please take the time and correct me. I made the experience that is quite important to use the proper numbers and terminology otherwise certain methods, and results are difficult to understand or do not make sense at all.

nik39 is a noob when it comes to video? that's funny.

with regard to #2, you are correct. this is why there is no "crop filter" in fcp. you must crop as a final step to the output file.
#3.c - fcp will show you absolute frame number for any given point in time.

here's that link i was talking about where they make 5 directories, 1 for each cadence. again, using mencoder to do the reverse telecine:

www.dvxuser.com

Is there any way to do it the other way around? For a certain frame I would like to know the exact time position.

yes it would do that to. just type in a + symbol, then the exact frame number you want to go to, then look at the little box that shows you the smpte timecode in hh:mm:ss;ff format.

there is no "place" to put the +frames into. just start typing and it'll go there. make sure the timeline or viewer window is active. or the canvas if you prefer to do it there. basically, any window but the browser will do this.

Thanks, a-swift.

Do you know of any commandline tile which can show a list of all frames incl. the stats (at least time-position... )?

tile=tool

i'd have to think about that one nik.

would it be fairly safe to say that most music videos of American artists are originally film or NTSC???

im based in Australia so there are some DVDs which I can buy which are PAL

The majority of videos shot in the US that originate on video camera (not film), will be shot with 23.976 fps. Lots of reasons why directors choose that frame rate. Most importantly, the ability to derive the other frame rates from this source and also the light needed to light scenes is less than higher frame rates.

ok cool - thanks - good to know

\

damn - does this mean i cant get away from windows? :)

The first crop parameters (usually only 8-12 pixels) is just to remove the overscan compensation that PO/ScreenPlay/etc apply to the videos. In the world of computers, we don't need to worry about overscan, so I want to get the picture to be the exact size of the frame.

I resize to 640 first for two reasons.
1: We are working in square pixel environments with computers, so 640 just works better IMO.
2: When I resize to 640, it also makes cropping for 16:9 easier, since all I have to do is take 60 pixels off the top and 60 pixels off the bottom, which results in a perfect 1.78:1 AR file (640x360).

3. Decimate.



That's exactly what IVTC is. IVTC is the removal of that duplicate frame. If I was leaving that duplicate frame in, it would just be considered decombing. Also, technically this is 23.976 fps, so that's why the timing matches up (that's a standard for pretty much all film content for final delivery -- for example, Blu-ray movies are encoded at 23.976 as well).



I can't answer that, never messed with VFR personally. I can't imagine that working well in VSL either? I could be wrong though :)



I've found that MOST music videos (at least before very recently) have a broken cadence. Lately, with the move to everything digital in the entire chain, it seems they're editing in true 24p and then applying the pulldown afterwards, so there are some with perfect cadences. But, again, this method handles broken cadences pretty well (IMO). Other than a few jumpy frames (literally a few, less than 1 second), it finds the cadence again and continues IVTC correctly.



We're all noobs :) That's why I'm glad we can have these discussions!

Haha, that's your call :) Personally my workflow has always been better in Windows because I learned everything I know in Windows. I have nothing against Macs (I use a Mac every day, and use a MBP for DJing), I just unfortunately don't have much knowledge when it comes to video tools in that environment.

I'd put my mac tools against anything in windows. There's not much I can't solve with MPEGStreamclip, FCP, Compressor, After Effects, Shake, ffmpeg and mencoder.

thats what i like to hear! i just want to be completely windows free

Again, I'm not doubting any tools on the Mac side. Just saying I learned on Windows, and so far, everything has worked for me.

using A-Swifts outstanding FCP Sessions tutorial as a guide, i've followed his steps in terms of using MPEG StreamClip to create a MOV file to edit in FCP, and also demux to get the m2v file to use the Reconnect media step.

However, following this thread, it would seem as though i would need to IVTC the file that I will eventually Reconnect to - right?

If so - i cant seem to use the 'Cinema Tools Reverse Telecine' option in FCP/Cinema Tools as it seems as though it cant read a M2V file - what should i do?

or i am doing it all wrong?

When doing this to videos can you see a noticeable difference?

You can absolutely see a noticeable difference - playback is much smoother and judder-free. It also reduces strain on your CPU since you are only decoding the frames that need to be decoded (instead of a bunch of duplicates). Depending on how you're deinterlacing videos right now, this could also lead to better picture quality since you aren't blending fields which some deinterlacers do.

Brett.... WOW... The difference in video quality at dublin was utterly amazing last night. Each of the distributed screens in the bar looked like a DVD image. I had non of the vertical tracing that I was getting before. The file size is quite reasonable.

Props to Smashvidz... their product quality was awesome!!!

Thanks

Cool now I just need to figure out how to do this.

ditto

it's not too hard if you follow the directions. You may come across some different scenarios though. depends on your source of videos.

I just want to say thanks for what is one of the most informative threads I've seen here.

This page from the Handbrake Wiki might help some: trac.handbrake.fr

So my question is for some very old stuff pulled from DVD's: it seems like for some stuff, there's no real way around deinterlacing?

Two videos I've struggled with are Clash "Rock The Casbah" and Rick James "Give It To Me Baby". Besides both of them being of pretty low quality in general, none of the detelecining/reverse telecining options in Compressor, Final Cut Pro, FCP's Cinema Tools, or Handbrake (which supposedly auto-detects cadence) have been able to successfully work on those two.

Am I missing something?

maybe these videos do not have 3:2 pulldown added so there is nothing to remove. maybe they were shot ntsc broadcast format natively. not likely but a possibility.

use compressor.

im trying to! But just completely not sure on what settings to use - if i use compressor to convert a m2v file to a m2v file (and selecting reverse telecine under the Frame Controls section) - does this mean that there is no generation loss?

If so - im still confused as to what file format to use QuickTime Movie, and select Uncompressed 10-bit in the Compression Type section? I thought this would put the M2V file through a conversion process which i dont want.

so confused!

as far as i know, the reverse telecine process on all the methods listed above will recompress each frame unless you use uncompressed as your export type.

can someone post up the .avs file. Seems easier than trying to type everything in the youtube vid.

Glad to hear it man!

LoadPlugin("X:\Path_to_DGIndex_Folder\DGDecode.dll")
LoadPlugin("X:\Path_to_Plugins_Folder\TDeint.dll")
MPEG2Source("video.d2v")
TDeint()
Decimate(5)
Crop(left,top,-right,-bottom)
Lanczos4Resize(640,360)
Crop(0,60,-0,-60)
ConvertToYV12()

Notes: Only use Decimate if the video is actually 3:2 pulldown, and only use Crop(0,60,-0,-60) if video is widescreen.

sorry if this is a stupid question but when you resize to 640x360... doesn't that cut off the chyrons on a promo only .vob?

That's cropping, resizing does just that it resizes the entire video.

LIke taking a 5x7 and shrinking it to a 3x5, It's the same picture just on a smaller scale.

sorry. i meant to say doesn't cropping cut off the chyrons on a letterbox promo only .vob? does that bother you esthetically?

thanks a million a-swift - i think i got it worked out now and seems to look good. You're help has been invaluable - thanks dude!

and thanks DJ Brett B for opening our eyes to this - im definitely always keen to hear about techniques to make my videos look sharper!

Brett, i was testing avisynth 2.6 and was able to utilize TDeint() without the LoadPlugin call.

Ah, yeah, if you have the TDeint.dll in your AviSynth plugins folder, you don't need to load it.

It bothers me much less esthetically than improper aspect ratios, which is what you'd get if you just play a letterboxed 4:3 video on a 16:9 monitor. Everything that I do is assuming that the video will be played on a 16:9 monitor. It's a good idea to start producing ALL of your edits and encodes in 16:9 from here on out - if you aren't already playing on 16:9 systems, eventually everything will be 16:9.

aaahhh...... im back to not liking my videos

after I IVTC the m2v file, and then use this to render to MP4. Most of the frames look beautiful but at random points in my video there is a lot of interlaced frames. It doesnt happen at regular intervals, and the length of interlaced frames changes each time so theres no pattern.

when i render to MP4 and also Deinterlace, the video looks pixelated and 'digital'.

i cant seem to get this right! anyone got any suggestions?