Cropping black borders on videos

tamarok★

Mar 04, 2023 - edited Mar 04, 2023 - permalink

We aren't a big fan big black borders on images or videos, so for anyone who finds themselves with a video of that type and is comfortable with ffmpeg:

To calculate the cropping rectangle:

ffmpeg -i in.mp4 -vf cropdetect  out.mp4

It will give an output a bit like:

[Parsed_cropdetect_0 @ 0x7fb18da04080] x1:564 x2:1355 y1:0 y2:1079 w:784 h:1072 x:568 y:4 pts:228864 t:14.900000 crop=784:1072:568:4
[Parsed_cropdetect_0 @ 0x7fb18da04080] x1:564 x2:1355 y1:0 y2:1079 w:784 h:1072 x:568 y:4 pts:229376 t:14.933333 crop=784:1072:568:4
[Parsed_cropdetect_0 @ 0x7fb18da04080] x1:564 x2:1355 y1:0 y2:1079 w:784 h:1072 x:568 y:4 pts:229888 t:14.966667 crop=784:1072:568:4
[Parsed_cropdetect_0 @ 0x7fb18da04080] x1:564 x2:1355 y1:0 y2:1079 w:784 h:1072 x:568 y:4 pts:230400 t:15.000000 crop=784:1072:568:4

and then using the detected rectangle, which is indicated in the output:

ffmpeg -i in.mp4 -filter:v "crop=784:1072:568:4" out.mp4

While we are okay with this sort of cropping, please avoid screen recordings or resizing the video. Any videos that we consider upscaled will either get merged into the smaller version or get removed.

Some more info here: https://blog.otterbro.com/ffmpeg-cropping-and...

tamarok★

Mar 04, 2023 - permalink

Deleted by tamarok★

Chainer★

Mar 05, 2023 - permalink

I could have the site do this automatically on upload, but chipperpip really hates reencoded videos...

Another thing that would be valuable is removing the black tiktok screen that takes up several seconds at the end of some videos, but this seems harder to automate.

chipperpip★

Mar 05, 2023 - edited Mar 05, 2023 - permalink

I'm not inherently opposed to it, but cropping is always going to be a lossy process, and a lot of these Instagram videos and such aren't the best quality to begin with, so I usually only consider it worth it for really egregious borders (that take up like 1/3 or more of the frame).

Trying to keep the quality loss minimal while cropping also increases the filesize, and the necessary settings may not be possible within the 25MB upload limit.

In the case of your second command to do the actual crop, you would want to add a Constant Rate Factor to avoid degrading the video too much (the lower the number the better), ideally something like:

-crf 7

...which is usually only usable for fairly short/small videos. I generally try to avoid going above 10 or so.

In terms of using -vf cropdetect, that does seem easier than my usual method of exporting a snapshot from the video and drawing bounding boxes to figure out the crop box! I tested it on this:

And got:

crop=1072:608:4:656

Which while it got the vertical parameters perfect, cuts off a few pixels on the sides for some reason instead of leaving it as 1080 horizontal. It doesn't actually matter in this case since her arms aren't going out of frame, but I'd have to test it some more before I was comfortable relying on it. EDIT: Oh I see, that was probably because 1080 isn't divisible by 16. That default setting for "round" is dumb, it needs to be set to 2.

I should note that I don't like using FFmpeg command lines manually, so I always recommend the FFmpeg Batch interface. You can still just throw "-vf cropdetect" into the parameters box by itself and use the "Try on console" button to see the suggested crop parameters (FFmpeg Batch also has an autocrop function, but it seems to be broken and just crashes for me).

Another thing that would be valuable is removing the black tiktok screen that takes up several seconds at the end of some videos, but this seems harder to automate.

Unlike cropping, trimming the length of a video (like removing the Tiktok screen, although that particular case is best done by just downloading the logoless original version of the video from Tiktok's servers using Snaptik or something) can be done losslessly without re-encoding the video data, assuming you get lucky with where the keyframes are placed. I'd agree trying to automate that process probably isn't worth the effort, especially since wonky keyframe placement can sometimes require human judgement on where to cut.

I'm not inherently opposed to it, but cropping is always going to be a lossy process, and a lot of these Instagram videos and such aren't the best quality to begin with, so I usually only consider it worth it for really egregious borders (that take up like 1/3 or more of the frame).

Trying to keep the quality loss minimal while cropping also increases the filesize, and the necessary settings may not be possible within the 25MB upload limit.

In the case of your second command to do the actual crop, you would want to add a Constant Rate Factor to avoid degrading the video too much (the lower the number the better), ideally something like:

```
-crf 7
```

...which is usually only usable for fairly short/small videos.  I generally try to avoid going above 10 or so.

In terms of using -vf cropdetect, that does seem easier than my usual method of exporting a snapshot from the video and drawing bounding boxes to figure out the crop box!  I tested it on this:

[[video_rVpGbZ]]

And got:

```
crop=1072:608:4:656
```

Which while it got the vertical parameters perfect, cuts off a few pixels on the sides for some reason instead of leaving it as 1080 horizontal.  It doesn't actually matter in this case since her arms aren't going out of frame, but I'd have to test it some more before I was comfortable relying on it.  EDIT:  Oh I see, that was probably because 1080 isn't [divisible by 16](https://www.bogotobogo.com/FFMpeg/ffmpeg_cropdetect_ffplay.php).  That default setting for "round" is dumb, it needs to be set to 2.

I should note that I don't like using FFmpeg command lines manually, so I always recommend the [FFmpeg Batch](https://ffmpeg-batch.sourceforge.io/) interface.  You can still just throw "-vf cropdetect" into the parameters box by itself and use the "Try on console" button to see the suggested crop parameters (FFmpeg Batch also has an autocrop function, but it seems to be broken and just crashes for me).

&nbsp;

> Another thing that would be valuable is removing the black tiktok screen that takes up several seconds at the end of some videos, but this seems harder to automate.

Unlike cropping, trimming the length of a video (like removing the Tiktok screen, although that particular case is best done by just downloading the logoless original version of the video from Tiktok's servers using [Snaptik](https://snaptik.app/) or something) can be done losslessly without re-encoding the video data, assuming you get lucky with where the keyframes are placed.  I'd agree trying to automate that process probably isn't worth the effort, especially since wonky keyframe placement can sometimes require human judgement on where to cut.