This project is read-only.
1
Vote

errors reported (1.5.15.0)

description

I have encoded a short Clip with Vidcoder 1.5.15.0 and in the end it says under completed in the row status:
Errors reported
but if I look into the log file I don't find any errors in it (excluding that it checks for bluray files that don't exist in transportstreams (which are from receivers)
## VidCoder 1.5.15.0 (x86)
# Starting job 1/1
#   Path: C:\Users\Siggi\Desktop\Der Schuh des Manitu.ts
#   Title: 1
#   Range: All
# Worker ready: Service state is Opened on pipe VidCoderWorker.f7f43706-e6a8-478e-b72b-2013cefbf454
# Connecting to process 5980 on pipe VidCoderWorker.f7f43706-e6a8-478e-b72b-2013cefbf454
[17:34:41] hb_init: starting libhb thread
[17:34:41] CPU: Intel(R) Core(TM)2 Quad CPU    Q9400  @ 2.66GHz
[17:34:41]  - logical processor count: 4
[17:34:41] OpenCL device #1: NVIDIA Corporation GeForce GTX 285
[17:34:41]  - OpenCL version: 1.0 CUDA
[17:34:41]  - driver version: 332.21
[17:34:41]  - device type:    GPU
[17:34:41]  - supported:      no
[17:34:41] Intel Quick Sync Video support: no
[17:34:41] hb_scan: path=C:\Users\Siggi\Desktop\Der Schuh des Manitu.ts, title_index=1
libbluray/bdnav/index_parse.c:162: indx_parse(): error opening C:\Users\Siggi\Desktop\Der Schuh des Manitu.ts/BDMV/index.bdmv
libbluray/bdnav/index_parse.c:162: indx_parse(): error opening C:\Users\Siggi\Desktop\Der Schuh des Manitu.ts/BDMV/BACKUP/index.bdmv
libbluray/bluray.c:1725: nav_get_title_list(C:\Users\Siggi\Desktop\Der Schuh des Manitu.ts) failed (06b9efe8)
[17:34:41] bd: not a bd - trying as a stream/file instead
libdvdnav: Using dvdnav version 4.1.3
libdvdread: Encrypted DVD support unavailable.
libdvdnav:DVDOpenFileUDF:UDFFindFile /VIDEO_TS/VIDEO_TS.IFO failed
libdvdnav:DVDOpenFileUDF:UDFFindFile /VIDEO_TS/VIDEO_TS.BUP failed
libdvdread: Can't open file VIDEO_TS.IFO.
libdvdnav: vm: failed to read VIDEO_TS.IFO
[17:34:41] dvd: not a dvd - trying as a stream/file instead
[17:34:41] file is MPEG Transport Stream with 188 byte packets offset 0 bytes
[17:34:41] Found the following PIDS
[17:34:41]     Video PIDS : 
[17:34:41]       0x1ff type H.264 (0x1b) (PCR)
[17:34:41]     Audio PIDS : 
[17:34:41]       0x203 type AC3 (0x81)
[17:34:41]     Subtitle PIDS : 
[17:34:41]     Other PIDS : 
[17:34:41] stream id 0x203 (type 0x81 substream 0x0) audio 0x203
[17:34:42] scan: decoding previews for title 1
[17:34:42] scan: audio 0x203: AC-3, rate=48000Hz, bitrate=384000 Deutsch (AC3) (5.1 ch)
[17:34:42] scan: 10 previews, 1920x1080, 25.000 fps, autocrop = 106/106/2/2, aspect 16:9, PAR 1:1
[17:34:42] stream: 19 good frames, 0 errors (0%)
[17:34:42] libhb: scan thread found 1 valid title(s)
hb_init_opencl_env: OpenCL support not available
[17:34:43] 1 job(s) to process
[17:34:43] OpenCL: Get context info failed
[17:34:43] OpenCL: Unable to get list of devices in context.
[17:34:43] work: failed to initialize OpenCL environment, using fallback
[17:34:43] starting job
[17:34:43] yadif thread started for segment 0
[17:34:43] yadif thread started for segment 1
[17:34:43] yadif thread started for segment 2
[17:34:43] yadif thread started for segment 3
[17:34:43] decomb filter thread started for segment 0
[17:34:43] decomb filter thread started for segment 2
[17:34:43] decomb filter thread started for segment 3
[17:34:43] decomb check thread started for segment 0
[17:34:43] decomb filter thread started for segment 1
[17:34:43] decomb check thread started for segment 1
[17:34:43] decomb check thread started for segment 2
[17:34:43] mask erode thread started for segment 0
[17:34:43] mask filter thread started for segment 0
[17:34:43] sync: expecting 3410 video frames
[17:34:43] mask dilate thread started for segment 2
[17:34:43] job configuration:
[17:34:43]  * source
[17:34:43] mask filter thread started for segment 2
[17:34:43]    + C:\Users\Siggi\Desktop\Der Schuh des Manitu.ts
[17:34:43]    + title 1, chapter(s) 1 to 1
[17:34:43]  * destination
[17:34:43]    + C:\Users\Siggi\Desktop\Der Schuh des Manitu.mp4
[17:34:43]    + container: MPEG-4 (avformat)
[17:34:43]  * video track
[17:34:43]    + decoder: h264
[17:34:43] mask erode thread started for segment 1
[17:34:43]      + bitrate 200 kbps
[17:34:43]    + filters
[17:34:43]      + Detelecine (pullup) (default settings)
[17:34:43] mask erode thread started for segment 2
[17:34:43]      + Decomb (default settings)
[17:34:43]      + Framerate Shaper (0:27000000:1080000)
[17:34:43]        + frame rate: same as source (around 25.000 fps)
[17:34:43]      + Crop and Scale (1280:580:106:106:2:2)
[17:34:43]        + source: 1920 * 1080, crop (106/106/2/2): 1916 * 868, scale: 1280 * 580
[17:34:43] decomb check thread started for segment 3
[17:34:43]    + dimensions: 1280 * 580, mod 0
[17:34:43]    + encoder: H.264 (x264)
[17:34:43]      + x264 preset: fast
[17:34:43]      + x264 tune: film
[17:34:43] mask filter thread started for segment 3
[17:34:43]      + h264 profile: high
[17:34:43]      + quality: 21.00 (RF)
[17:34:43]  * audio track 1
[17:34:43] mask erode thread started for segment 3
[17:34:43]    + decoder: Deutsch (AC3) (5.1 ch) (track 2, id 0x203)
[17:34:43]      + bitrate: 384 kbps, samplerate: 48000 Hz
[17:34:43]    + mixdown: Stereo
[17:34:43] mask dilate thread started for segment 0
[17:34:43]    + encoder: AAC (avcodec)
[17:34:43]      + bitrate: 160 kbps, samplerate: 48000 Hz
[17:34:43] mask dilate thread started for segment 3
[17:34:43] mask filter thread started for segment 1
[17:34:43] mask dilate thread started for segment 1
[17:34:43] file is MPEG Transport Stream with 188 byte packets offset 0 bytes
[17:34:43] reader: first SCR 304507029 id 0x1ff DTS 304640742
[17:34:43] encx264: encoding at constant RF 21.000000
[17:34:43] encx264: unparsed options: ref=2:deblock=-1,-1:weightp=1:subme=6:psy-rd=1.00,0.15:rc-lookahead=30
x264 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.1 Cache64
x264 [info]: profile High, level 3.1
[17:34:43] h264: "Chapter 1" (1) at frame 0 time 18000
[17:34:43] sync: first pts is 18000
[17:38:20] hb_ts_stream_decode - eof
[17:38:20] reader: done. 1 scr changes
[17:38:25] work: average encoding speed for job is 16.712875 fps
[17:38:25] sync: got 3673 frames, 3410 expected
[17:38:25] decomb: deinterlaced 613 | blended 336 | unfiltered 2722 | total 3671
[17:38:25] render: lost time: 10800 (0 frames)
[17:38:25] render: gained time: 10800 (4 frames) (0 not accounted for)
[17:38:25] h264-decoder done: 3673 frames, 0 decoder errors, 0 drops
x264 [info]: frame I:20    Avg QP:17.79  size: 56974
x264 [info]: frame P:1197  Avg QP:20.64  size: 17776
x264 [info]: frame B:2452  Avg QP:22.81  size:  4675
x264 [info]: consecutive B-frames:  3.0% 18.0% 17.2% 61.8%
x264 [info]: mb I  I16..4:  5.9% 75.7% 18.4%
x264 [info]: mb P  I16..4:  0.8%  5.8%  0.9%  P16..4: 44.3% 21.3% 11.3%  0.0%  0.0%    skip:15.7%
x264 [info]: mb B  I16..4:  2.7%  3.4%  0.3%  B16..8: 23.6%  4.7%  0.2%  direct:15.7%  skip:49.5%  L0:36.9% L1:53.3% BI: 9.8%
x264 [info]: 8x8 transform intra:62.9% inter:65.8%
x264 [info]: coded y,uvDC,uvAC intra: 56.5% 66.0% 21.4% inter: 20.1% 25.7% 0.3%
x264 [info]: i16 v,h,dc,p: 37% 19% 28% 16%
x264 [info]: i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 19% 13% 29%  4%  7%  8%  6%  6%  6%
x264 [info]: i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 34% 16% 12%  6%  8%  8%  6%  6%  4%
x264 [info]: i8c dc,h,v,p: 57% 16% 21%  6%
x264 [info]: Weighted P-Frames: Y:4.0% UV:2.7%
x264 [info]: ref P L0: 63.3% 36.7%
x264 [info]: ref B L0: 75.6% 24.4%
x264 [info]: ref B L1: 93.7%  6.3%
x264 [info]: kb/s:1845.40
[17:38:25] mux: track 0, 3669 frames, 33880694 bytes, 1843.85 kbps, fifo 2048
[17:38:25] mux: track 1, 6891 frames, 2943989 bytes, 160.22 kbps, fifo 4096
[17:38:25] stream: 7353 good frames, 0 errors (0%)
[17:38:25] libhb: work result = 0
# Encode succeeded but error(s) were reported during the encode.
# Job completed (Elapsed Time: 3m 45s)

comments

RandomEngy wrote Jan 15, 2014 at 7:09 PM

It's usually easier to pick out the errors from the Log window because they are colored red there. I see a few messages the system might have considered to be an error.

Djfe wrote Jan 15, 2014 at 8:27 PM

do you have an idea where they come from or should I make another encode and then take a screenshot of the log window?

RandomEngy wrote Jan 15, 2014 at 9:13 PM

Generally if your encode completes, but has errors things will be OK. Various components can report errors but I don't know all of the potential messages. You can post a screenshot if you want but I believe in your ability to find red text.

Djfe wrote Jan 17, 2014 at 5:03 PM

Well, I wasn't able to spot red text at first, because I clicked on "open log" in the context menu of the completed file instead of opening window -> log
maybe you should change that VidCoder opens specific log files only in notepad which has no syntax highlighting at all ;)

anyway this was the issue:
hb_init_opencl_env: OpenCL support not available
[17:34:43] 1 job(s) to process
[17:34:43] OpenCL: Get context info failed
[17:34:43] OpenCL: Unable to get list of devices in context.
[17:34:43] work: failed to initialize OpenCL environment, using fallback
in the general log file there was the following line added at the end (which is not in the log file of a specific encoded file):
ERROR: hb_init_opencl_env: OpenCL support not available
Do (older) Nvidia cards only support CUDA and not OpenCL?
[17:34:41] OpenCL device #1: NVIDIA Corporation GeForce GTX 285
[17:34:41]  - OpenCL version: 1.0 CUDA
[17:34:41]  - driver version: 332.21
[17:34:41]  - device type:    GPU
[17:34:41]  - supported:      no
Nvidia themself says that the GTX 285 supports even OpenCL up to Version 3.0/3.1
TEXT

I'm using the newest stable driver (332.21)

Is this an issue in Handbrake itself?

RandomEngy wrote Jan 17, 2014 at 5:40 PM

Any detection of OpenCL capabilities will be handled by HandBrake. I'll investigate and decide whether or not VidCoder should explicitly suppress this "error" as it's just letting you know that it's falling back to non-hardware-accelerated methods.

s55 wrote Jan 18, 2014 at 4:26 PM

Nivida Cards are not supported with the OpenCL code. AMD and Intel only.

s55 wrote Jan 18, 2014 at 4:31 PM

Also OpenGL is different to OpenCL. Nvidia cards support OpenCL 1.1. 1.2 and later is not supported.
Also, it's some kind of layer on tip of CUDA, so there will likely be overhead due to this and probably won't support ZeroCopy so all you'll get is reduced quality and slower encodes.

Djfe wrote Jan 18, 2014 at 7:17 PM

Ahh I see now, thanks for pointing that out ;)
I searched with Google for OpenCL and gtx 285 and it returned this page
so I didn't read it completely and thought it was OpenCL (G and C look very similar and I just "scanned" the page)

Well, I hope they will at least work on CUDA after they released a stable OpenCL code :)

s55 wrote Jan 19, 2014 at 4:10 PM

There are no plans to support CUDA. The OpenCL code may eventually be supported on Nvidia cards but it;s not currently being worked on.

Also note, it;s only 5% performance improvement anyway, along with a minor loss in quality.

Djfe wrote Jan 21, 2014 at 7:13 PM

There are no plans, doesn't mean it won't happen in the future, but possibly an integration of HEVC/H.265 is going to have a much a higher priority, once there will be an open soucre encoder

why do you think it will bring a loss in quality?
OpenCL doesn't bring quality loss eather and it is not a hardware decoder, it is a software decoder that runs on graphic hardware
I don't know what zerocopy is (clone data!?) but I don't think that nvidia would only implement the standard partially

as far as I know OpenCL was developed by Apple, specific for Nvidia cards

or is ZeroCopy part of a higher standard/new OpenCL Version?

just found this: https://devtalk.nvidia.com/default/topic/452602/is-there-a-quot-zero-copy-quot-equivalent-in-opencl-and-how-to-use-it-in-opencl-/
sounds different to me... (maybe you are not up to date)

s55 wrote Jan 21, 2014 at 9:01 PM

There are no plans, doesn't mean it won't happen in the future, but possibly an integration of HEVC/H.265 is going to have a much a higher priority, once there will be an open soucre encoder
CUDA has been already rejected it. We won't be accepting any patches for it. We may accept good patches for OpenCL filters if they provide sufficient improvement.

There is already an x265 project with a working encoder, and there is already a patch to integrate this into HandBrake. Currently there is no support in MP4 or MKV muxers so it's raw h265 stream out only. In other words, it'll be a while before we make this publicly available. It's also horribly slow still even on ultra high end hardware. It'll improve over time though if they keep at it.
why do you think it will bring a loss in quality?
OpenCL doesn't bring quality loss eather and it is not a hardware decoder, it is a software decoder that runs on graphic hardware
HandBrake has 2 scalers now. Lanczos (in software) and BiCubic (in OpenCL). Lanczos is a better quality scaler. If MCW had implemented a lanczos version, there should in theory have been minimal difference.

Also, some algorithms don't directly port to OpenCL and in porting, those changes that need to be made can have a negative effect on quality.
as far as I know OpenCL was developed by Apple, specific for Nvidia cards
No, it wasn't developed specifically for Nvidia cards. AMD and IBM where both part of the development group also along with Apple.
just found this: https://devtalk.nvidia.com/default/topic/452602/is-there-a-quot-zero-copy-quot-equivalent-in-opencl-and-how-to-use-it-in-opencl-/
sounds different to me... (maybe you are not up to date)
CUDA has the same concept i believe in later versions, but on Nvidia Cards, OpenCL can't use that feature currently on their cards. They've been lagging behind on the OpenCL bandwagon, my guess, is they are too busy promoting CUDA.

The problem is, neither CUDA or OpenCL on GPU's are particularly suitable for video encoders currently. The algorithms you implement need to scale massively, and many of them simply don't or can't. This is why x264 only implemented Lookhead in OpenCL. It's one of the few areas that would actually scale to a large enough number of threads to get a benefit from OpenCL.

What we are seeing now, is Nvidia and Intel putting dedicated hardware in instead, that's custom designed just for video encoding and they are both achieving much higher quality and speed. It's not matching x264 in terms of quality or filesize, but it's into the realm of usable for many people. AMD has started jumping on this as well but they seem to be a bit behind at the moment. They seem to make making pretty significant improvements each year, particularly QuickSync so that's a much better area to concentrate on.

Djfe wrote Jan 21, 2014 at 11:16 PM

OK, I understand
you seem to be one of the developers behind handbrake
have you heard about Parallela by Adapters (it is a Kickstarter project)
they claim to make a supercomputer for everyone
which is kinda true, but maybe is a little bit exaggerated for now
they are planning to scale their chips in the future to over 1000 cores
what do you think about it in terms of encoding?
Would so many cores speed up the image/movement/etc. analysis considerably or are openly the CPU architecture and frequency important?

s55 wrote Jan 23, 2014 at 9:31 PM

They are not really suited for video encoding. They'll have the same kind of issues as GPUs which are effectively the same idea, thousands of tiny compute units vs a small amount of large ones. For some things this is great, for others no so much.

If the algorithms don't scale to thousands of threads, there is no point in having a chip that can run thousands of threads if your algorithm can only make use of 16 at a time. Your better off with 16 beefy cores than 1000 tiny cores.


If you look at x264 at the moment, if you scale up to 16 threads, there is actually a small amount of quality loss in there. In a crude example, when you split the picture up into units of work, you encode say 32x32 blocks. However, if you encode 64x64 blocks, you can make better predictions because you have a wider scope of what the pixels to the left / right / above / below of your units are. However you can only split a picture up into a relativity small number of chunks.

What we've are trying to do in HandBrake is find bits (like scaling and filters) we can offload without too big an affect on quality so that CPU time is freed up for x264. The scaler (which is about 5% of the CPU time in HandBrake, takes around 40% of my AMD 7850 GPU). Not very power efficient but nevermind :)


Dedicated video encoding hardware or improved multimedia extensions (such as the recently introduced AVX2) is what we want. If i recall correctly, the x264 guys managed to get around 10% improvement with CPU's running AVX2 (Haswell or the latest AMD parts)

Djfe wrote Jan 23, 2014 at 10:42 PM

interesting
though as far as I know the Parallela doesn't behave like a gpu and you can compute different stuff on each core -> you can do more than with gprs because you aren't fixed to one algorithm
therefore I thought it would be able to compare movements and such things (movement analysis)
and as it has a lot of course you can compute more stuff at the same time

have to look up avx2 now :)

I think one of the next Intel generations takes us to 8 or more real cores in the consumer market which leads to 16 and more virtual cores (-> will be a rewarding investigation :) )

Djfe wrote Jan 23, 2014 at 10:50 PM

wait... did the x264 guys manage to get their hands on the excavator architecture by AMD or how were they able to test AVX2?
Haswell is already released but excavator will not be released probably before 2015

s55 wrote Jan 24, 2014 at 5:51 PM

It may be AVX on AMD and AVX2 for haswell. They do get parts from AMD, but if they get pre-release parts they'll be under NDA so we have no way of knowing what they actually have. Checking wikipedia it looks like it's Intel Only currently.

x264 really flies on the 6-Core I7's and 8 Core Xeons. Sadly these are still quite pricey.
though as far as I know the Parallela doesn't behave like a gpu and you can compute different stuff on each core
While that may be true, you've still got to split your work up into lots of different jobs and that's a very non-trivial task. Sometimes you just need large fast compute units. It gets more interesting when you pair Large and small together and have shared memory, you start opening up some opportunities. The Arm Cortex chip that they join with the Paralella is pretty weak.

I think you'd spend a year or two trying to get something working by which time it'll have been replaced by something new. Not exactly practical for a hobbyist open-source project.


Fwiw, if you want more speed, you can adjust the x264 preset to a faster setting.
Just as an example, Using UltraFast preset for a DVD source can top 1000fps on my machine.
Going to placebo preset drops the speed down to a ~ 6 - 8 fps
Obviously your making a trade off doing that so it's good to find a middle ground that suits your personal needs. HandBrake tends to air towards better quality/filesize than speed by default with most of the presets.

Djfe wrote Jan 25, 2014 at 12:08 AM

well as far as I know slower encoding speeds tend to make the file smaller but also reduce quality slightly
while faster encoding speeds end up in a bigger file size and slightly better quality

while we are at it (sorry for switching the topic): do you have an explanation why "Very fast" is such an outlier and seems to work kinda unpredictable according to the following article?
http://mattgadient.com/2014/01/06/handbrake-rf-slower-speeds-craziness/

I'm glad you have such a fast computer :)
I'm still using an Intel Quad-core (before i7 and stuff) and 32bit win7
but I'm going to buy one soon
(after Haswell-E, ddr4,g-sync,sata-express/name got released)

s55 wrote Jan 25, 2014 at 5:12 PM

well as far as I know slower encoding speeds tend to make the file smaller but also reduce quality slightly
while faster encoding speeds end up in a bigger file size and slightly better quality
Slower presets turn on more features which result in better compression and for the most part, wont' decrease quality. Sometimes you might see a larger file because a particular feature being enabled for example, might preserve an aspect of the video better. (i.e grain)


Not sure what his deal with Very Fast is. There is a lot that can have an effect so it's not guaranteed to be a curve. Different types of videos are affected differently by different features. http://mewiki.project357.com/wiki/X264_Settings < Lots of good info here. Also http://dev.beandog.org/x264_preset_reference.html Not sure if it's fully up to date.
I'm glad you have such a fast computer :)
It's a Haswell 4770K Quad. Going to a 6-Core would improve things again. I tend to use slower settings normally. Using Ultrafast preset isn't really usable in the end result.