Note: this article has nothing to do with video, releases or whatever else you usually care about, I’m going to talk about typesetting. If you don’t know what it is or don’t care – stop reading right now.
As you might know, there’s the haruhichan guide to 10bit, which is apparently one of the most popular playback guides lately. It’s more or less right but there’s one problem – the author (nand) suggests you to use internal MPC-HC renderer (ISR for short) to get the most awesome 10bit playback.
After getting a couple of complains about our bakemonogatari typesetting and reading some derping about broken TS on different forums, I decided to at least explain things to nand since he seemed adequate and might want to update his guide. I joined CGi channel and showed him two test samples on which ISR used to fail.
“If you’re analpained about test cases, aspect ratios or watch anime with shitty typesetting, use a different subtitle renderer like xy-VSFilter. Be warned though, the subtitles will be very blurry as a result. I recommend waiting until it supports madVR’s overlay hooks like MPC-HC does. Alternatively, don’t use subtitles (thanks tp7 for the suggestion)”
…What? I was a bit surpised with this stupid update and figured I’ll explain things to you myself.
Nowadays, there are three major subtitle renderers:
- Libass (mplayer2).
- VSFilter and its forks (threaded-vsfilter, xy-vsfilter).
- Internal MPC-HC VSFilter, which is essentialy original VSFilter with some modifications.
But let’s start with some theory.
In a simple case, your video playback routine looks like this (1920×1200 monitor, 720p video):
Note that there’s one important difference – subtitle renderer works before video renderer in the case of libass/VSFilter, while ISR works after video is rendered. What does it mean? Quite much.
Libass and VSFilter most likely merge subs into video while it still has its original resolution and non-RGB colorspace (this works like described unless you’re converting your video to RGB in the decoder or use ffdshow’s resize functions. You shouldn’t do it – this is a renderer’s job and madvr does it perfectly). Then renderer resizes your video without caring about subtitles being on it and converts it to RGB.
Note: this is completely true only for xy-vsfilter. Original VSFilter still converts the video to RGB, renders the subs and then converts it back to YV12. This doesn’t affect any of the following comparisons, though.
ISR, however, works on a resized (if resolution of your screen != resolution of the video) video with black bars added (if aspect ratio of your screen != AR of the video), which is already converted to RGB. Thus it needs to resize subtitles to your screen resolution and renders them in RGB to correctly blend them with video. And this is where all the fun starts.
First of all, I won’t speak much about libass. It’s good and all, but has its own problems and AFAIK there’s not a single one fully working DirectShow realization. Just look at the OP pic and you’ll get the idea how awesome libass works with UTW’s Amagami SS Plus Ep01.
Not all ass tags are resizable (or at least easily resizable). Here is a sample video that shows you how ISR fails to resize \fry and \frx (basic 3d-rotation tags). Just run it with ISR and see for yourself.
But well, according to nand I’m being “analpained about test cases” so here are some more examples. From our PPD release:
|Original ISR||Resized ISR||Resized VSFilter|
Again, ISR is broken.
But but but… maybe it’s just our shitty typesetting? Okay, possible. More fails:
- UTW’s Ano Hana, Ep03 (typeset by __ar):
- Chibiki’s The Idolm@ster, Ep25 (typeset by Daiz):
Original resolution ISR
One more time:
If you still think that it’s only about shitty typesetting – please, find me someone better than these guys (Note: I’m not sure if it’s possible).
I hope you got the point – many of 3d rotations will be broken for you with ISR.
One more problem about resizing – anamorphic video. VSFilter, so as libass, applies subs to the video before it’s resized to display resolution, while ISR goes its own way and does the same thing after. The result is different horizontal scaling of subtitles:
Yet again, this can easily break all typesetting if the subs were typeset with VSFilter behavior in mind (REC for some reason doesn’t have any typesetting and I don’t have any other anamorphic video right now, so no _real_ examples).
But that’s not all. You can notice that on the second resized ISR screenshot of iM@S some letters are placed outside the video frame. And this is yet another problem with this broken piece of software – it can’t clip subs to the original picture resolution, so it actually renders subs even if they’re outside the video frame. Here’s a sample from our Maria Holic release, and one more example of __ar’s Ano Hana typesetting:
|Original resolution||Resized ISR|
And no, your favourite argument about “Position subtitles relative to the video frame” setting in ISR won’t work – it doesn’t help in this situation.
I wish I could stop here, but there is one more problem. I didn’t mentioned RGB conversion for nothing, it’s yet another thing that works differently in VSFilter and ISR. Here’s a sample of qIIq/THORA bakemonogatari typesetting, or some screenshots from it:
As you might know, there are basically two different colormatrices which are used during YUV->RGB conversion: Rec601 and Rec709. First one is almost always used for SD content, second one – for HD video. The problem lies in when this conversion happens – in the case of correctly configured playback, it is done in a video renderer (e.g. MadVR), thus VSFilter (so as libass) applies subs to original YUV stream, and then renderer does its job (converting both subs and original video as a single frame using one colormatrix), while ISR merges subs with the final RGB picture. This is important for HD content: if the subs were typesetted to Rec601 video – they’ll work just fine with VSFilter while ISR will give you wrong colors since it doesn’t care about colormatrices and uses the colors exactly how they were defined in the subs.
Note: this is a little funny, but Aegisub with ffvideosource video provider outputs HD video using Rec709 colormatrix – thus colors will actually work right in ISR, but end up broken in libass/VSFilter. However, I think most of typesetters know this problem and use one of the ways to get Rec601 video even for HD stuff (like, opening the video via avisynth script with ConvertToRGB(“Rec601″) line in it, or using patched ffms2 builds). This was already reported to ffms2 and Aegi devs and I hope will be changed in the future.
But there’s one important thing – since VSFilter applies subs in video resolution before it was resized to your screen resolution, the subs will have lower resolution compared to ISR, in other words: will be blurry. This, for some reason, seems to bug a lot of people, but I actually don’t see any problems here. The sharpness highly depends on your video renderer’s resizer, and as a wise guy said – most subtitles might look too sharp if rendered at 1080p when the video is 720p. Here’s a simple comparison made by Daiz:
Sure, this is somewhat more important for SD content, but yet again, it depends on many factors – your video renderer, used fonts, border and background colors. And I guess this was the main reason why Nyaa nowadays allows 576p upscales of DVD content (not to mention that typesetting of 480p video is a little painful).
- If you’re fine with broken 3d rotations, broken clipping, broken colors, lags on some heavy typesetting but want your subs to be sharp as much as possible – use ISR. But don’t recommend it to others and don’t complain – not a single typesetter in his right mind would guarantee compatibility of his typesetting with ISR. Again: not a single one.
- If you want subs to actually work right – use VSFilter. The best of available forks is xy-vsfilter (the one author of this article uses, so as many other cool guys :) ). It’s basically VSFilter with some great speed improvements.
- You should stfu about subtitle renderers if you know nothing about typesetting
Note: here’s one interesting thread about the new subtitle renderer interface, so things might actually change in the future and hopefully ISR will finally die.