Commit Graph

70 Commits (main)

Author SHA1 Message Date
Tim Rogers 1ca6ff8ca5
Fixed crash when supplying FFMPEG_BINARY environment variable (#30022) 4 weeks ago
Matt Jankowski edde54e991
Update stoplight to version 4.1.0 (#28366) 2 months ago
Claire 93957daa50
Fix error when processing remote files with unusually long names (#28823) 4 months ago
Claire 06444c86c7
Fix division by zero in video in bitrate computation code (#27129) 8 months ago
Eugen Rochko 0f3f9b611f
Change video bitrate to always fit within size limit (#26970) 8 months ago
Eugen Rochko 01b87a1632
Change video compression parameters (#26631) 9 months ago
yufushiro 3aac12981c
Fix unexpected audio stream transcoding when uploaded video is eligible to passthrough (#26608)
Co-authored-by: Claire <claire.github-309c@sitedethib.com>
9 months ago
Christian Schmidt f2257069b2
Fix AVIF attachments (#26264) 10 months ago
Claire dc8f1fbd97
Merge pull request from GHSA-9928-3cp5-93fm
* Fix attachments getting processed despite failing content-type validation

* Add a restrictive ImageMagick security policy tailored for Mastodon

* Fix misdetection of MP3 files with large cover art

* Reject unprocessable audio/video files instead of keeping them unchanged
11 months ago
Nick Schonning c66250abf1
Autofix Rubocop Regex Style rules (#23690)
Co-authored-by: Claire <claire.github-309c@sitedethib.com>
12 months ago
Matt Jankowski 5e060e1f44
Fix Performance/Sum cop (#24788) 1 year ago
Nick Schonning 0cfdd1a401
Enable Rubocop Style/StringConcatenation defaults (#23792) 1 year ago
Nick Schonning af4c95100c
Autofix Rubocop Style/FormatString (#23743) 1 year ago
Nick Schonning d2dcb6c45a
Autofix Rubocop Style/UnpackFirst (#23741) 1 year ago
Nick Schonning 2177daeae9
Autofix Rubocop Style/RedundantBegin (#23703) 1 year ago
Nick Schonning a6f77aa28a
Autofix Rubocop Lint/AmbiguousOperatorPrecedence (#23681) 1 year ago
Nick Schonning 669f6d2c0a
Run rubocop formatting except line length (#23632) 1 year ago
Pierre Bourdon 36bc90e8aa
blurhash_transcoder: prevent out-of-bound reads with <8bpp images (#20388)
The Blurhash library used by Mastodon requires an input encoded as 24
bits raw RGB data. The conversion to raw RGB using Imagemagick did not
previously specify the desired bit depth. In some situations, this leads
Imagemagick to output in a pixel format using less bpp than expected.
This then manifested as segfaults of the Sidekiq process due to
out-of-bounds read, or potentially a (highly noisy) memory infoleak.

Fixes #19235.
2 years ago
Claire 9d4861b498
Remove dependency on running Redis server for db:setup (#18560) 2 years ago
Holger 39b489ba4c
fix: `s3_force_single_request` not parsed (#17922) 2 years ago
Claire 166f6e4b50
Fix some media attachments being converted with too high framerates (#17619)
Video files with variable framerates are converted to constant framerate videos
and the output framerate picked by ffmpeg is based on the original file's
container framerate (which can be different from the average framerate).

This means that an input video with variable framerate with about 30 frames per
second on average, but a maximum of 120 fps will be converted to a constant 120
fps file, which won't be processed by other Mastodon servers.

This commit changes it so that input files with VFR and a maximum framerate
above the framerate threshold are converted to VFR files with the maximum frame
rate enforced.
2 years ago
Claire 48f8658d34
Fix upload of remote media with OpenStack Swift sometimes failing (#16998)
Under certain conditions, files fetched from remotes trigger an error when
being uploaded using OpenStack Swift. This is because in some cases, the
remote server will not return a content-length, so our ResponseWithLimitAdapter
will hold a `nil` value for `#size`, which will lead to an invalid value
for the Content-Length header of the Swift API call.

This commit fixes that by taking the size from the actually-downloaded file
size rather than the upstream-provided Content-Length header value.
3 years ago
Claire 6ba8bc45cb
Add S3_FORCE_SINGLE_REQUEST env var to work around S3 compatibility issues (#16866)
Fixes #16822
3 years ago
Claire fc3ae1343d
Switch from unmaintained paperclip to kt-paperclip (#16724)
* Switch from unmaintained paperclip to kt-paperclip

* Drop some compatibility monkey-patches not required by kt-paperclip

* Drop media spoof check monkey-patching

It's broken with kt-paperclip and hopefully it won't be needed anymore

* Fix regression introduced by paperclip 6.1.0

* Do not rely on pathname to call FastImage

* Add test for ogg vorbis file with cover art

* Add audio/vorbis to the accepted content-types

This seems erroneous as this would be the content-type for a vorbis stream
without an ogg container, but that's what the `marcel` gem outputs, so…

* Restore missing for_as_default method

* Refactor Attachmentable concern and delay Paperclip's content-type spoof check

Check for content-type spoofing *after* setting the extension ourselves, this
fixes a regression with kt-paperclip's validations being more strict than
paperclip 6.0.0 and rejecting some Pleroma uploads because of unknown
extensions.

* Please CodeClimate

* Add audio/vorbis to the unreliable set

It doesn't correspond to a file format and thus has no extension associated.
3 years ago
Takeshi Umeda 818e0b314f
Fix unsupported video error message handling (#16581) 3 years ago
rinsuki 5ed5f62705
Fix animated GIF generates animated thumbnail (#16216) 3 years ago
Claire a5f91a11d0
Fix older migrations on Ruby 3 (#16174) 3 years ago
Claire 566fc90913
Add Ruby 3.0 support (#16046)
* Fix issues with POSIX::Spawn, Terrapin and Ruby 3.0

Also improve the Terrapin monkey-patch for the stderr/stdout issue.

* Fix keyword argument handling throughout the codebase

* Monkey-patch Paperclip to fix keyword arguments handling in validators

* Change validation_extensions to please CodeClimate

* Bump microformats from 4.2.1 to 4.3.1

* Allow Ruby 3.0

* Add Ruby 3.0 test target to CircleCI

* Add test for admin dashboard warnings

* Fix admin dashboard warnings on Ruby 3.0
3 years ago
Eugen Rochko 036556d350
Fix media processing getting stuck on too much stdin/stderr (#16136)
* Fix media processing getting stuck on too much stdin/stderr

See thoughtbot/terrapin#5

* Remove dependency on paperclip-av-transcoder gem

* Remove dependency on streamio-ffmpeg gem

* Disable stdin on ffmpeg process
3 years ago
Shubhendra Singh Chauhan c8d11b8bdb
Fixed code quality issues (#15541)
* Added .deepsource.toml

* Removed bad use of `alias`

* Fixed operand order in the binary expression

* Prefixed unused method arguments with an underscore

* Replaced the old OpenSSL algorithmic constants with the newer strings initializers.

* Removed unnecessary UTF-8 encoding comment
3 years ago
luigi 087ed84367
Optimize map { ... }.compact calls (#15513)
* Optimize map { ... }.compact

using Enumerable#filter_map, supported since Ruby 2.7

* Add poyfill for Enumerable#filter_map
3 years ago
abcang efffdd3778
Fix rubocop config and warnings (#15503)
* disable NewCops

* update TargetRubyVersion

* Fix Lint/MissingSuper for ActiveModelSerializers::Model

* Fix Lint/MissingSuper for feed

* Fix Lint/FloatComparison

* Do not use instance variables
3 years ago
Eugen Rochko 1045549f85
Add stoplight for object storage failures, return HTTP 503 (#13043) 3 years ago
ThibG 5bbc9a4f78
Fix downloading remote media files when server returns empty filename (#14867)
Fixes #14817
4 years ago
Eugen Rochko e6b272e5c9
Change REST API to return empty data for suspended accounts (#14765) 4 years ago
dependabot[bot] 8972e5f7f6
Bump rubocop from 0.86.0 to 0.88.0 (#14412)
* Bump rubocop from 0.86.0 to 0.88.0

Bumps [rubocop](https://github.com/rubocop-hq/rubocop) from 0.86.0 to 0.88.0.
- [Release notes](https://github.com/rubocop-hq/rubocop/releases)
- [Changelog](https://github.com/rubocop-hq/rubocop/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rubocop-hq/rubocop/compare/v0.86.0...v0.88.0)

Signed-off-by: dependabot[bot] <support@github.com>

* Fix for latest RuboCop

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Yamagishi Kazutoshi <ykzts@desire.sh>
4 years ago
ThibG 0ebc8d8626
Fix thumbnail color extraction (#14464)
* Fix contrast calculation for thumbnail color extraction

Luminance calculation was using 0-255 RGB values instead of 0-1 sRGB values,
leading to incorrectly-computed contrast values.

Since we use ColorDiff already, just use its XYZ colorspace conversion code
to get the value.

* Require at least 3:1 contrast for both accent and foreground colors

* Lower required contrast for the accent color
4 years ago
ThibG a1412491b7
Change content-type to be always computed from file data (#14452)
* Change content-type to be always computed from file data

Restore previous behavior, detecting the content-type isn't very
expensive, and some instances may serve files as application/octet-stream
regardless of their true type, making fetching media from them fail, while
it used to work pre-3.2.0.

* Add test
4 years ago
Takeshi Umeda 101485a41f
Fix mimetype returning nil (#14356) 4 years ago
ThibG 701e5b9a19
Fix ogg vorbis files with a cover art not being correctly processed (#14255) 4 years ago
Eugen Rochko 99f3a55540
Add color extraction for audio thumbnails (#14209) 4 years ago
Eugen Rochko 534da9ba23
Fix audio uploads without embedded image (#14203) 4 years ago
Eugen Rochko 7aaf2b44ec
Fix remote files not using Content-Type header, streaming (#14184) 4 years ago
Eugen Rochko 64aac30733
Add customizable thumbnails for audio and video attachments (#14145)
- Change audio files to not be stripped of metadata
- Automatically extract cover art from audio if it exists
- Add `thumbnail` parameter to `POST /api/v1/media`, `POST /api/v2/media` and `PUT /api/v1/media/:id`
- Add `icon` to represent it in attachments in ActivityPub
- Fix `preview_url` containing URL of missing missing image when there is no thumbnail instead of null
- Fix duration of audio not being displayed on public pages until the file is loaded
4 years ago
ThibG 346d9b3d2e
Add blurhash to preview cards (#13984)
Fixes #13001
4 years ago
ThibG 927f9ea499
Fix GifReader exceptions (#13760) 4 years ago
Eugen Rochko c3ca3801f2
Add separate cache directory for non-local uploads (#12821) 4 years ago
Eugen Rochko 0773a512ac
Fix trying to delete already deleted file when post-processing (#13406)
Fix #13403
4 years ago
ThibG 6c79b7237e
Fix Paperclip using deprecated URI.escape function (#13320)
Monkey-patch Paperclip to perform URL escaping in a slightly more
appropriate way, and get rid of runtime deprecation warnings.
4 years ago
Eugen Rochko 0c8945e5ff
Change `tootctl media remove-orphans` to work for all classes (#13316)
Change `tootctl media lookup` to not use an interactive prompt
4 years ago