Commit Graph

572 Commits

Author SHA1 Message Date
Akihiko Odaki
40e5d2303b Validate HTTP response length while receiving (#6891)
to_s method of HTTP::Response keeps blocking while it receives the whole
content, no matter how it is big. This means it may waste time to receive
unacceptably large files. It may also consume memory and disk in the
process. This solves the inefficency by checking response length while
receiving.
2018-03-26 14:02:10 +02:00
Akihiko Odaki
b2a4ffd3a9 Change columns in notifications nonnullable (#6764) 2018-03-24 12:51:28 +01:00
Akihiko Odaki
54b273bf99 Close http connection in perform method of Request class (#6889)
HTTP connections must be explicitly closed in many cases, and letting
perform method close connections makes its callers less redundant and
prevent them from forgetting to close connections.
2018-03-24 12:49:54 +01:00
Eugen Rochko
da70aca28e
Restore username validation to disallow dots, for now (#6863)
Usernames with dots in them do not work with routes, because the dot usually separates the desired page format (e.g. json). I don't want to mess with changing route constraints for this patch release.
2018-03-22 11:30:22 +01:00
ThibG
6f531d140b Fix MENTION_RE to not match nil usernames (#6862) 2018-03-22 10:45:48 +01:00
Eugen Rochko
93897134ca
Permit dots in usernames with conditions (#6844)
* Permit dots in usernames with conditions

- Dot cannot be the start or end of username
- a.lice and al.ice are considered the same during sign-up

* Fix regex mixin flags
2018-03-21 10:26:53 +01:00
Alexander
0306e3e9be bugfixes and gem update (#6831)
* update to new version of devise_pam_authenticatable2

* fix behaviour if suffix is nil, fix environment loading, fix user email creation

* code cleanup/fix linter warning
2018-03-19 20:08:56 +01:00
Daniel Hunsaker
4a0a19fe54 Handle Mastodon::HostValidationError when pulling remoteable assets (#6782)
This will prevent, for example, `rake mastodon:redownload_avatars` from crashing when an instance is no longer responding to connection attempts, instead silently continuing as expected.
2018-03-17 13:27:50 +01:00
Eugen Rochko
1aaec701bb
Fix #6715: Make catalan words with the L geminate letter work in hashtags (#6741) 2018-03-11 14:55:49 +01:00
Renato "Lond" Cerqueira
9aba44ea79 Rescue when there's no extension in the remotable (#6358)
* Rescue when there's no extension in the remotable
Sometimes the remotable is pointing to a directory with no file
extension. Maybe it should not be expecting to identify based on
extensions to begin with, but since it's the case, it should be ready
for it.

* Fix codeclimate issue

* Check if filename is nil instead of rescueing exception
Suggestion made in the PR

* Avoid concatenation issue if filename is nil
If filename is nil, extname was undefined

* Invert condition
Address PR comments
2018-03-08 02:25:10 +01:00
Eugen Rochko
cfa9b6e13a
Remove text requirement when media attached from statuses (#6672) 2018-03-07 08:28:52 +01:00
Effy Elden
dd9d00d293 Add additional first_name and last_name SAML attribute statement options, and modify Omniauthable concern to use full_name or first_name + last_name if not available (#6669) 2018-03-07 06:19:10 +01:00
Eugen Rochko
78d772af86
Fix #3807: Increase avatars to 400x400 max (#6651)
But do not upscale when they are smaller
2018-03-06 06:29:01 +01:00
Akihiko Odaki
c110fa62ac Provide default OTP_SECRET value for development environment (#6617) 2018-03-04 20:28:24 +01:00
Eugen Rochko
9110db41c5
Federate pinned statuses over ActivityPub (#6610)
* Federate pinned statuses over ActivityPub

* Display pinned toots in web UI

Fix #6117

* Fix migration

* Fix tests

* Update outbox_serializer.rb

* Update remove_serializer.rb

* Update add_serializer.rb

* Update fetch_featured_collection_service.rb
2018-03-04 09:19:11 +01:00
Eugen Rochko
47bdb9b33b
Fix #942: Seamless LDAP login (#6556) 2018-02-28 19:04:53 +01:00
Eugen Rochko
41a01bec23
Federated reports (#6570)
* Fix #2176: Federated reports

* UI for federated reports

* Add spec for ActivityPub Flag handler

* Add spec for ReportService
2018-02-28 06:54:55 +01:00
Ian McCowan
c33931b613 Fix prev/next links on public profile page (#6497)
* Fix prev/next links on public profile page

* Don't make pagination urls if no available statuses

* Fix empty check method

* Put left chevron before prev page link

* Add scope for pagination "starting at" a given id

* Status pagination try 2:

s/prev/older and s/next/newer
"older" on left, "newer" on right
Use new scope for "newer" link
Extract magic 20 page size to constant
Remove max_id from feed pagination as it's not respected

* Reinstate max_id for accounts atom stream

* normalize
2018-02-26 03:31:28 +01:00
Eugen Rochko
5cc716688a
Ensure the app does not even start if OTP_SECRET is not set (#6557)
* Ensure the app does not even start if OTP_SECRET is not set

* Remove PAPERCLIP_SECRET (it's not used by anything, actually)

Imports are for internal consumption and the url option isn't even
used correctly, so we can remove the hash stuff from them
2018-02-26 01:31:44 +01:00
Ghislain Loaec
e668180044 New variable OAUTH_REDIRECT_AT_SIGN_IN + Ref #6538 (not only SAML strategies) (#6540) 2018-02-23 01:16:17 +01:00
Ghislain Loaec
3084fe4959 New env variable: SAML_SECURITY_ASSUME_EMAIL_IS_VERIFIED + fixes #6533 (#6538) 2018-02-22 23:31:25 +01:00
Eugen Rochko
90f12f2e5a
Focal points (#6520)
* Add focus param to media API, center thumbnails on focus point

* Add UI for setting a focal point

* Improve focal point icon on upload item

* Use focal point in upload preview

* Add focalPoint property to ActivityPub

* Don't show focal point button for non-image attachments
2018-02-22 00:35:46 +01:00
Eugen Rochko
d3a62d2637
Fix #6525: Make sure file is opened in LazyThumbnail processor (#6529) 2018-02-22 00:28:19 +01:00
Eugen Rochko
61ed133fea
Account archive download (#6460)
* Fix #201: Account archive download

* Export actor and private key in the archive

* Optimize BackupService

- Add conversation to cached associations of status, because
  somehow it was forgotten and is source of N+1 queries
- Explicitly call GC between batches of records being fetched
  (Model class allocations are the worst offender)
- Stream media files into the tar in 1MB chunks
  (Do not allocate media file (up to 8MB) as string into memory)
- Use #bytesize instead of #size to calculate file size for JSON
  (Fix FileOverflow error)
- Segment media into subfolders by status ID because apparently
  GIF-to-MP4 media are all named "media.mp4" for some reason

* Keep uniquely generated filename in Paperclip::GifTranscoder

* Ensure dumped files do not overwrite each other by maintaing directory partitions

* Give tar archives a good name

* Add scheduler to remove week-old backups

* Fix code style issue
2018-02-21 23:21:32 +01:00
Eugen Rochko
a7171af0a3
Fix avatar and header issues by using custom geometry detector (#6515)
* Fix avatar and header issues by using custom geometry detector

Revert a part of #6508. The file passed to dynamic styles method
was not actually a file, but an instance of Paperclip::Attachment,
which broke all styles by always returning {} from the method.

One problem with GIF avatars was that Paperclip::GeometryDetector
reported wrong dimensions for them, e.g. 120x120 GIF avatar would
for some reason be detected as 120x53. By writing our own geometry
parser, we can use FastImage, which also happens to be faster than
ImageMagick, to detect image dimensions, which are also correct.

Unfortunately, this PR does not implement skipping a `convert`
entirely if the dimensions are already correct, as I found no easy
way to write that behaviour into Paperclip without rewriting the
Paperclip::Thumbnail class.

* Only invoke convert if dimension or format needs to be changed
2018-02-21 03:40:12 +01:00
Eugen Rochko
66105929e0
Don't resize avatars/headers if their dimensions are already right (#6508)
Also don't apply "-quality 80" option which is probably the reason
for slight color differences between original and remote image
(because it would apply it twice, once on original instance, and
again on the receiving instance)
2018-02-19 16:06:12 +01:00
Eugen Rochko
cba2897108
Cache relationships in API (#6482)
* Cache relationships in API

* Fetch relationships for search results in UI

* Only save one account's maps in each cache item
2018-02-18 03:14:46 +01:00
Eugen Rochko
a71af98401
Push discovered status through streaming API within a time window (#6484)
Time window of 6 hours
2018-02-17 14:28:48 +01:00
Eugen Rochko
9dbae6e8a1
Save video metadata and improve video OpenGraph tags (#6481)
* Save metadata from video attachments, put correct dimensions into OG tags

* Add twitter:player for videos

* Fix code style and test
2018-02-16 07:22:20 +01:00
Eugen Rochko
3ebc0ad4d3
Full-text search for authorized statuses (#6423)
* Add full-text search for authorized statuses

- Search API will return statuses that match the query
- Only for logged in users
- Only if you are author of the status,
- Or you were mentioned in it
- Or you favourited or reblogged it
- Configuration over `ES_ENABLED`, `ES_HOST`, `ES_PORT`, `ES_PREFIX`
- Run `rails chewy:deploy` to create & populate index

Fix #5880
Fix #4293
Fix #1152

* Add commented out docker-compose configuration for ES container

* Optimize index import, filter search results

* Add basic normalization to the index

* Add better stemming and normalization to the index

* Skip webfinger request if search query includes both @ and a space

* Fix code style

* Visually separate search result sections

* Fix code style issues
2018-02-09 23:04:47 +01:00
Eugen Rochko
76f3d5d16b
Add preference to always display sensitive media (#6448) 2018-02-09 00:26:57 +01:00
abcang
298c81c00f Clear account cache of notification target_status (#6442) 2018-02-08 15:33:23 +01:00
Akihiko Odaki
67f7ffa792 Change user_id column non-nullable (#6435) 2018-02-07 16:35:44 +01:00
Eugen Rochko
38e0133e1b
Make PAM gem optional, allow configuration over environment (#6415) 2018-02-04 15:05:53 +01:00
Renato "Lond" Cerqueira
4e4f1b0dcb Add option to show only local toots in timeline preview (#6292)
* Add option to show only local toots in timeline preview
Right know, toots from all the known fediverse are shown in the main
page of an instance. That however doesn't reflect the instance itself.
With this option the admin may choose to display only local toots so
that users checking the instance get a better idea of internal
conversations.

* Fix issues pointed by codeclimate and eslint

* Add default message for community timeline

* Update pl.yml
2018-02-04 06:00:10 +01:00
Eugen Rochko
26f21fd5a0
CAS + SAML authentication feature (#6425)
* Cas authentication feature

* Config

* Remove class_eval + Omniauth initializer

* Codeclimate review

* Codeclimate review 2

* Codeclimate review 3

* Remove uid/email reconciliation

* SAML authentication

* Clean up code

* Improve login form

* Fix code style issues

* Add locales
2018-02-04 05:42:13 +01:00
Alexander
04fef7b888 pam authentication (#5303)
* add pam support, without extra column

* bugfixes for pam login

* document options

* fix code style

* fix codestyle

* fix tests

* don't call remember_me without password

* fix codestyle

* improve checks for pam usage (should fix tests)

* fix remember_me part 1

* add remember_token column because :rememberable requires either a password or this column.

* migrate db for remember_token

* move pam_authentication to the right place, fix logic bug in edit.html.haml

* fix tests

* fix pam authentication, improve username lookup, add comment

* valid? is sometimes not honored, return nil instead trying to authenticate with pam

* update devise_pam_authenticatable2 and adjust code. Fixes sideeffects observed in tests

* update devise_pam_authenticatable gem, fixes for codeconventions, fix finding user

* codeconvention fixes

* code convention fixes

* fix idention

* update dependency, explicit conflict check

* fix disabled password updates if in pam mode

* fix check password if password is present, fix templates

* block registration if account is maintained by pam

* Revert "block registration if account is maintained by pam"

This reverts commit 8e7a083d650240b6fac414926744b4b90b435f20.

* fix identation error introduced by rebase

* block usernames maintained by pam

* document pam settings better

* fix code style
2018-02-02 10:18:55 +01:00
Eugen Rochko
1cc44cba81
Fix #6331 (#6341)
UserTrackingConcern is circumvented by SessionsController#create
because it calls warden, which calls the User#update_tracked_fields!
method directly. Move returning user logic to that method.
2018-01-23 20:52:30 +01:00
Akihiko Odaki
613e7c7521 Rename ResolveRemoteAccountService to ResolveAccountService (#6327)
The service used to be named ResolveRemoteAccountService resolves local
accounts as well.
2018-01-22 14:25:09 +01:00
Yamagishi Kazutoshi
238de58e65 Change belongs_to_required_by_default to true (#5888) 2018-01-19 20:56:47 +01:00
Eugen Rochko
d799921c75
Replace tutorial modal with welcome e-mail (#6273)
* Remove onboarding modal

* Welcome e-mail

* Send welcome e-mail after confirmation

* Remove obsolete translations
2018-01-18 19:17:25 +01:00
Eugen Rochko
ed867eca9d
Move e-mail digest task to sidekiq, reduce workload, improve hint (#6252) 2018-01-15 04:34:28 +01:00
Eugen Rochko
dbda87c31f
Revert #5772 (#6221) 2018-01-08 10:57:52 +01:00
Patrick Figel
5ec25ff3e1 Fix email confirmation link not updating email (#6187)
A change introduced in #6125 prevents
`Devise::Models::Confirmable#confirm` from being called for existing
users, which in turn leads to `email` not being set to
`unconfirmed_email`, breaking email updates. This also adds a test
that would've caught this issue.
2018-01-05 00:15:35 +01:00
Akihiko Odaki
161c72d66d Allow to dereference Follow object for ActivityPub (#5772)
* Allow to dereference Follow object for ActivityPub

* Accept IRI as object representation for Accept activity
2018-01-03 18:08:57 +01:00
Patrick Figel
04ecf44c2f Add confirmation step for email changes (#6071)
* Add confirmation step for email changes

This adds a confirmation step for email changes of existing users.
Like the initial account confirmation, a confirmation link is sent
to the new address.

Additionally, a notification is sent to the existing address when
the change is initiated. This message includes instruction to reset
the password immediately or to contact the instance admin if the
change was not initiated by the account owner.

Fixes #3871

* Add review fixes
2018-01-02 16:55:00 +01:00
Eugen Rochko
38fc1b498d
Add more instance stats APIs (#6125)
* Add GET /api/v1/instance/peers API to reveal known domains

* Add GET /api/v1/instance/activity API

* Make new APIs disableable, exclude private statuses from activity stats

* Fix code style issue

* Fix week timestamps
2017-12-29 19:52:04 +01:00
Eugen Rochko
ad75ec8b5b
Add filters to admin UI for custom emojis (#6003) 2017-12-13 13:28:31 +01:00
Eugen Rochko
20a6584d2d
Clean up admin UI for accounts (#6004)
* Add staff filter to admin UI for accounts, remove obsolete columns

* Only display OStatus section in admin UI for accounts when OStatus data
2017-12-13 12:15:10 +01:00
Akihiko Odaki
14c4a33cd9 Change account_id non-nullable in lists (#5979) 2017-12-12 04:11:17 +01:00