Commit Graph

429 Commits

Author SHA1 Message Date
Matt Jankowski
8ec8410651 After remote follow worker specs (#3098)
* Add specs for the after remote follow workers

* Refactor the after remote follow workers
2017-05-18 21:10:41 +02:00
Eugen Rochko
1efda1c453 Fix #2572 - Resolve preview cards for remote statuses as well as local ones (#3088) 2017-05-17 00:41:15 +02:00
alpaca-tc
682b68438e Improve Account#triadic_closures (#3079) 2017-05-16 12:06:38 +02:00
alpaca-tc
a2a2af244c Optimize Status#permitted_for 24x (#3069)
* Build query with arel node

* Add spec for current Status#permitted_for implementation

* Refactor status.rb

* Order by visibility to optimize query
2017-05-16 02:54:17 +02:00
Eugen Rochko
af706583bd Fix change of status callbacks not setting in_reply_to_account_id and (#3072)
possibly others when expected. Add some tests for it
2017-05-15 21:20:55 +02:00
Eugen Rochko
d0dd9eb5b5 Feature conversations muting (#3017)
* Add <ostatus:conversation /> tag to Atom input/output

Only uses ref attribute (not href) because href would be
the alternate link that's always included also.

Creates new conversation for every non-reply status. Carries
over conversation for every reply. Keeps remote URIs verbatim,
generates local URIs on the fly like the rest of them.

* Conversation muting - prevents notifications that reference a conversation
(including replies, favourites, reblogs) from being created. API endpoints
/api/v1/statuses/:id/mute and /api/v1/statuses/:id/unmute

Currently no way to tell when a status/conversation is muted, so the web UI
only has a "disable notifications" button, doesn't work as a toggle

* Display "Dismiss notifications" on all statuses in notifications column, not just own

* Add "muted" as a boolean attribute on statuses JSON

For now always false on contained reblogs, since it's only relevant for
statuses returned from the notifications endpoint, which are not nested

Remove "Disable notifications" from detailed status view, since it's
only relevant in the notifications column

* Up max class length

* Remove pending test for conversation mute

* Add tests, clean up

* Rename to "mute conversation" and "unmute conversation"

* Raise validation error when trying to mute/unmute status without conversation
2017-05-15 03:04:13 +02:00
Matt Jankowski
5f727f9068 Update rails to version 5.0.3 (#3054)
* Update rails to version 5.0.3

* Update aws-sdk to version 2.9.17

* Update capistrano to version 3.8.1

* Update climate_control to version 0.2.0

* Update debug_inspector to version 0.0.3

* Update et-orbi to version 1.0.4

* Update oj to version 3.0.7

* Update microformats2 to version 2.9.0

* Update lograge to version 0.5.1

* Clean up usage of Microformats2 object to remove deprecation warnings

* Update microformats2 to version 3.0.1
2017-05-14 16:30:14 +02:00
Eugen Rochko
657496b5a9 Do not cancel PuSH subscriptions after encountering "permanent" error… (#3046)
* Do not cancel PuSH subscriptions after encountering "permanent" error response

After talking with MMN about it, turns out some servers/php setups do
return 4xx errors while rebooting, so this anti-feature that was meant
to take load off of the hub is doing more harm than good in terms of
breaking subscriptions

* Update delivery_worker.rb
2017-05-14 03:22:48 +02:00
alpaca-tc
e9810cbad6 Fixes NoMethodError: undefined method 'first' for nil:NilClass (#3036) 2017-05-13 15:56:12 +02:00
Matt Jankowski
0d70fe2659 Spec coverage and refactor for pubsub/delivery worker (#3021)
* Framework for delivery worker spec

* Refactor of pubsub delivery worker
2017-05-12 20:35:36 +02:00
Eugen Rochko
5abdc77c80 Add conversation model, <ostatus:conversation /> (#3016)
* Add <ostatus:conversation /> tag to Atom input/output

Only uses ref attribute (not href) because href would be
the alternate link that's always included also.

Creates new conversation for every non-reply status. Carries
over conversation for every reply. Keeps remote URIs verbatim,
generates local URIs on the fly like the rest of them.

* Fix conversation migration

* More spec coverage for status before_create

* Prevent n+1 query when generating Atom with the new conversations

* Improve code style

* Remove redundant local variable
2017-05-12 19:09:21 +02:00
Eugen Rochko
b5a9c6b3d2 Prepend reblogs' wrapper content with "RT @original_author", (#3013)
so that when a reblog parse fails on another instance, it doesn't
look like a misattributed/stolen text
2017-05-12 17:46:44 +02:00
Matt Jankowski
60ecfb87ae Coverage for domain block worker (#2997) 2017-05-11 22:25:15 +02:00
Eugen Rochko
65f9db73b0 Fix #2027 - Accept own ID for remote follow with and without preceding @ (#2991)
* Fix #2027 - Accept own ID for remote follow with and without preceding @
Fix #2177 - Omit leading "acct:" in remote follow redirect template expansion

* Fix test
2017-05-11 21:54:30 +02:00
Jack Jennings
2d000e9c4e Add specs for current Status#permitted? implementation (#2982) 2017-05-11 11:25:10 +02:00
Eugen Rochko
72698bc3b4 Fix regressions from #2683 (#2970)
* Fix regressions from #2683

Properly format spoiler text HTML, while keeping old logic for blankness intact
Process hashtags and mentions in spoiler text
Format spoiler text for Atom
Change "show more" toggle into a button instead of anchor
Fix style regression on dropdowns for detailed statuses

* Fix lint issue

* Convert spoiler text to plaintext in desktop notifications
2017-05-11 00:28:10 +02:00
Matt Jankowski
08949cca41 Pubsub confirmation worker spec (#2974)
* Add specs for valid challenge response from pubsub confirmation worker

* Refactor the pubsub confirmation worker
2017-05-11 00:19:37 +02:00
Eugen Rochko
a231f915a0 Fix #2955 - Send HEAD request ahead of GET when fetching URL previews (#2972) 2017-05-10 23:30:07 +02:00
Matt Jankowski
cc9a6a710f Spec feed insert worker (#2965)
* Spec for feed insert worker when missing records

* more specs!

* Refactor feed insert worker
2017-05-10 20:32:05 +02:00
Matt Jankowski
fd66f7cdc0 Disable sidekiq logging in test mode (#2962) 2017-05-10 16:16:42 +02:00
Matt Jankowski
d142544159 Spec coverage and refactor of digest mailer worker (#2961) 2017-05-10 15:45:43 +02:00
Eugen Rochko
3a38322a54 Add spec for Pubsubhubbub::DistributionWorker. PuSH-deliver public items (#2954)
to all subscribers. IDN-normalize callback URLs for subscriptions on insert.
2017-05-10 02:55:43 +02:00
Matt Jankowski
682507bc3c Specs for pubsub subscribe service (#2951)
* Add spec for pubsubhubbub/subscribe

* Refactor pubsubhubbub/subscribe service
2017-05-09 20:48:30 +02:00
Matt Jankowski
441d6dc734 Spec and refactor for pubsubhubbub/unsubscribe service (#2946)
* Add coverage for pubsub unsubscribe service

* Refactor pubsub unsubscribe service
2017-05-09 19:58:18 +02:00
Akihiko Odaki
d5cabfe5c6 Use CLD3 (#2949)
Compact Language Detector v3 (CLD3) is the successor of CLD2, which was
used in the previous implementation. CLD3 includes improvements since CLD2,
and supports newer compilers. On the other hand, it has additional
requirements and cld3-ruby, the FFI of CLD3 for Ruby, is still new and may
be still inmature.

Though CLD3 is named after CLD2, it is implemented with a neural network
model, different from the old implementation, which is based on a Naïve
Bayesian classifier.

CLD3 supports newer compilers, such as GCC 6. CLD2 is not compatible with
GCC 6 because it assigns negative values to varibales typed unsigned.
(see internal/cld_generated_cjk_uni_prop_80.cc) The support for GCC 6 and
newer compilers are essential today, when some server operating system
such as Ubuntu Server 16.10 has GCC 6 by default.

On the one hand, CLD3 requires C++11 support. Environments with old
compilers such as Ubuntu Server 14.04 needs to update the system or install
a newer compiler.

CLD3 needs protocol buffers as a new dependency. However,it is not
considered problematic because major server operating systems, CentOS and
Ubuntu Server provide them.

The FFI cld3-ruby was written by me (Akihiko Odaki) for use in Mastodon.
It is still new and may be inmature, but confirmed to pass existing tests.
2017-05-09 19:58:03 +02:00
abcang
08e94d1b19 Fixed bug that timeline can not be displayed by InvalidURIError (#2947) 2017-05-09 18:17:41 +02:00
Matt Jankowski
5bea42412e Services specs for subscribe and unsubscribe (#2928)
* Add specs for unsubscribe service

* Fix non existent methods in unsubscribe service

* Clean up status handling in subscribe service
2017-05-09 00:45:02 +02:00
Matt Jankowski
04166c4a35 Specs for API push controller, with refactor (#2926)
* Coverage for api push controller

* Refactor the api/push controller
2017-05-09 00:44:30 +02:00
Matt Jankowski
7a7d12d27f Delegate Account#user_locale method and allow nil (#2927) 2017-05-08 23:10:50 +02:00
Matt Jankowski
bba537a7be Improve allowed language handling (#2897)
* Dont allow empty value in user allowed languages

* Sanitize language input to reject blank values in array
2017-05-08 03:32:52 +02:00
Matt Jankowski
b8ba719f73 Unblock domain service specs/refactor (#2867)
* Add spec for unblock domain service

* Refactor UnblockDomainService
2017-05-07 14:44:28 +02:00
alpaca-tc
74036a2c9d Hotfix convert string from symbol (#2856)
* Convert key to string from symbol

* Prefer :public_send instead of
2017-05-06 23:06:52 +02:00
alpaca-tc
05b72368ed Fixes slow query in notifications api (#2851)
* Add inclusion validation for activity_type

* When excluding all types with `WHERE NOT IN`, the query search all rows.
2017-05-06 21:55:40 +02:00
alpaca-tc
59804abc3d Optimize MuteService and AfterBlockService (#2836) 2017-05-06 16:31:07 +02:00
Eugen Rochko
b8e166894b Fix Scheduler::SubscriptionsScheduler (#2834)
* Fix Scheduler::SubscriptionsScheduler, add worker test for it

* Change production log level of Sidekiq to "warn" instead of "info"
2017-05-06 13:05:03 +02:00
Matt Jankowski
3f5b994ff0 Stream entry specs and refactor to use delegate (#2827)
* Add coverage for stream entry delegated methods

* Use delegate with allow_nil to clean up stream entry
2017-05-06 04:00:21 +02:00
Matt Jankowski
484c9709b6 Misc spec coverage improvements (#2821)
* Dont use raise_error by itself (avoids warning)

* Add coverage for AccountFilter

* Improve coverage and refactor for Subscription#lease_seconds

* Improve coverage and refactor for NotificationMailer

* Simplify assignment of min/max threshold on subscription
2017-05-05 14:56:00 -04:00
masarakki
d08f1112d5 use Twitter::Extractor for creating links (#2502) 2017-05-05 19:48:22 +02:00
Matt Jankowski
20c37ed0f9 Add specs (and refactor) of FetchRemoteResourceService and SearchService (#2812)
* Coverage for fetch remote resource service

* Refactor fetch remote resource service

* Coverage for search service

* Refactor search service
2017-05-05 17:26:04 +02:00
Eugen Rochko
81584779cb More robust PuSH subscription refreshes (#2799)
* Fix #2473 - Use sidekiq scheduler to refresh PuSH subscriptions instead of cron

Fix an issue where / in domain would raise exception in TagManager#normalize_domain

PuSH subscriptions refresh done in a round-robin way to avoid hammering a single
server's hub in sequence. Correct handling of failures/retries through Sidekiq (see
also #2613). Optimize Account#with_followers scope. Also, since subscriptions
are now delegated to Sidekiq jobs, an uncaught exception will not stop the entire
refreshing operation halfway through

Fix #2702 - Correct user agent header on outgoing http requests

* Add test for SubscribeService

* Extract #expiring_accounts into method

* Make mastodon:push:refresh no-op

* Queues are now defined in sidekiq.yml

* Queues are now in sidekiq.yml
2017-05-05 02:23:01 +02:00
Matt Jankowski
f9d398e8fb Remove the react-rails gem (#2801)
* Remove react-rails gem

* Fix broken view spec
2017-05-04 23:45:18 +02:00
alpaca-tc
74c8ca699c Delete records in smaller transaction (#2802) 2017-05-04 23:44:39 +02:00
Matt Jankowski
84eb425f38 Remove redis clear from before in spec (#2795) 2017-05-04 20:53:39 +02:00
Matt Jankowski
2bd132d458 Replace best_in_place editor on admin settings page (#2789)
* Remove best_in_place

* Replace best_in_place usage with rails helpers

* Move admin/settings#index to #edit action

* Remove click_to__edit from i18n
2017-05-04 18:12:44 +02:00
Eugen Rochko
13c16b4e95 Likely fix #2458, fix #2031 - handle out-of-order deletes for statuses (#2734)
* Likely fix #2458, fix #2031 - handle out-of-order deletes for statuses

If a delete arrives before the original status, cache that information
for 6h, and if the original status arrives in that window, ignore it

* Add test case
2017-05-04 04:34:57 +02:00
ThibG
8d4e7504b1 Additional specs for URI handling (#2759) 2017-05-03 20:40:14 +02:00
Eugen Rochko
bafd22ecf4 Fix #2706 - Always respond with 200 to PuSH payloads (#2733)
Fix #2196 - Respond with 201 when Salmon accepted, 400 when unverified
Fix #2629 - Correctly handle confirm_domain? for local accounts
Unify rules for extracting author acct from XML, prefer <email>, fall back
to <name> + <uri> (see also #2017, #2172)
2017-05-03 17:02:18 +02:00
Matt Jankowski
8c5ad23b24 Language improvements, replace whatlanguage with CLD (#2753)
* add failing en specs

* add cld2 gem

* Replace WhatLanguage with CLD
2017-05-03 16:59:31 +02:00
Eugen Rochko
f5bf5ebb82 Replace sprockets/browserify with Webpack (#2617)
* Replace browserify with webpack

* Add react-intl-translations-manager

* Do not minify in development, add offline-plugin for ServiceWorker background cache updates

* Adjust tests and dependencies

* Fix production deployments

* Fix tests

* More optimizations

* Improve travis cache for npm stuff

* Re-run travis

* Add back support for custom.scss as before

* Remove offline-plugin and babili

* Fix issue with Immutable.List().unshift(...values) not working as expected

* Make travis load schema instead of running all migrations in sequence

* Fix missing React import in WarningContainer. Optimize rendering performance by using ImmutablePureComponent instead of
React.PureComponent. ImmutablePureComponent uses Immutable.is() to compare props. Replace dynamic callback bindings in
<UI />

* Add react definitions to places that use JSX

* Add Procfile.dev for running rails, webpack and streaming API at the same time
2017-05-03 02:04:16 +02:00
Matt Jankowski
268dd32d76 Auth sign out (#2511)
* Add a spec for signing out

* Add spec showing that suspended user gets a 403 forbidden on sign out

* Allow suspended account users to sign out
2017-05-02 23:37:58 +02:00
ThibG
bea97ea766 Add rspec to further specify FollowRemoteAccountService (#2414) 2017-05-02 23:37:26 +02:00
Kaylee
7880671f35 Add option to disable two factor auth in admin accounts panel. (#2584)
* Add option to disable two factor auth in admin accounts panel.
Closes #2578

* Add @mjankowski's suggestions.
* Moves destroy actions behind User#disable_two_factor!
* Adds spec coverage for Admin:TwoFactorAuthenticationsController and User#disable_two_factor!
2017-05-02 21:07:12 +02:00
Yamagishi Kazutoshi
6f75c8451d Fix subscription expiration condition (#2715)
* Fix subscription expiration condition

* dry and add spec
2017-05-02 18:21:22 +02:00
Matt Jankowski
a4859446ab Coverage for remote follows (#2694)
* Add coverage for create with empty acct value

* Add coverage for create with webfinger failure

* Add coverage for create with webfinger providing bad values

* Add coverage for create when webfinger is good

* Add coverage for session[:remote_follow] having data

* Simplify how remote follow pulls acct from session

* Remote follow behaves more like model

* Move the discovery portions of remote follow out of controller

* Check for suspended accounts
2017-05-02 00:44:23 +02:00
Matt Jankowski
f025cc6782 Filter on allowed user language preferences (#2361)
* Naive approached to timeline filtering

* Convert allowed_languages into a db column

* Allow users to choose languages to see statuses in

* Style list items as two columns

* Add a hint to explain language filtering preference
2017-05-01 17:42:13 +02:00
Matt Jankowski
5259319cf5 Reports spec for media attachments, clean up method (#2660)
* Add coverage for Report#media_attachments

* Direct query on media attachment
2017-04-30 18:15:49 +02:00
Akihiko Odaki
a6788662b0 Revert "Use PostgreSQL inheritance for blocks and mutes (#2520)" (#2634)
This reverts commit 5135d609b7.
2017-04-30 08:49:24 -04:00
Matt Jankowski
fdcf884cf7 Extract user tracking into concern (#2600) 2017-04-30 00:28:16 +02:00
Akihiko Odaki
5135d609b7 Use PostgreSQL inheritance for blocks and mutes (#2520) 2017-04-30 00:27:31 +02:00
Matt Jankowski
f48cb3eb17 More coverage yes more even more (#2627)
* Add coverage for admin/confirmations controller

* Coverage for statuses controller show action

* Add coverage for admin/domain_blocks controller

* Add coverage for settings/profiles#update
2017-04-30 00:25:38 +02:00
Joël Quenneville
27301312a6 Add tests to the Feed model (#2594)
This adds a test for the `Feed#get` method. While the data
transformations in `Feed#get` may seem redundant, they are important to
maintain the order from Redis. The tests I wrote will fail if someone
tries to refactor away this "redundancy" (as I tried to do in the first
iteration of this change).
2017-04-29 00:21:35 +02:00
Patrick Figel
8ac7fca5d0 Set correct attachment type for rejected media (#2599)
In #2110, a new attachment type "unknown" was introduced for
attachments that were rejected due to a domain being blocked using
reject_media. However, the "type" field was never set to "unknown"
because a default value of "0" (image) is set for that column,
causing the `type.blank?` expression to always equal false.

This version uses type_changed? instead, causing the type to be set
to "unknown" unless a type has been explicitly set. This introduces
a small change in behaviour causing the type to be set to unknown
before paperclip calls `before_post_process`. Presumably this
behaviour is more appropriate than the current one because the
attachment type has not been determined by that point.

Included are new tests for `ProcessFeedService` and
`UpdateRemoteProfileService` which now check that remote media is
downloaded for non-blocked domains and is rejected for others.
2017-04-29 00:18:32 +02:00
Matt Jankowski
a823509b99 Fix broken view spec for about/links (#2591) 2017-04-28 17:54:03 +02:00
Matt Jankowski
9262f6968b Fix broken spec for about/links view partial (#2586) 2017-04-28 16:45:24 +02:00
Yamagishi Kazutoshi
71e73e36cd Fix spec for hide get started (#2585) 2017-04-28 16:44:57 +02:00
Matt Jankowski
9566893cc9 More controller specs (#2561)
* Add render_views in more places

* Delegate methods from account to user with allow nil true, so that admin accounts show view renders when missing a user

* Use actual account instances in authorize follow controller spec
2017-04-28 15:12:37 +02:00
Matt Jankowski
429480bb77 Return missing page when tag does not exist (#2563) 2017-04-28 15:11:21 +02:00
Matt Jankowski
effb08edbb More status specs (#2564)
* Add rough outline of coverage needed for public timeline

* Specs for visibility, replies, boosts

* Specs for silenced account

* Specs for local_only option

* Specs for blocks and mutes

* Add tentative spec around including other silenced account statuses

* Add with_public_visibility scope

* Add simple coverage for tag_timeline

* Tag timeline includes replies

* Replace tag.statuses with a tagged_with scope in tag timeline method

* Use with_public_visibility in tag timeline

* Extract common scope between public and tag timelines to method

* Extract local domain check to local_only scope

* Extract local_only check to starting scope method

* Move list of excluded from timeline account ids to account model

* Simplify excluded accounts list on account model

* Only join accounts when needed

* Rename method for account specific filtering

* Extract method for account exclusions

* Fix bug where silenced accounts were not including statuses from other silenced accounts

* DRY up filter application from account or no account

* timeline_scope can be private

* Add spec showing that account can find its excluded accounts ids

* Add spec which fails if local_only does not have a left outer join

* rubocop
2017-04-28 15:10:41 +02:00
Matt Jankowski
a39e719b39 Add render_views to admin/reports controller spec (#2548) 2017-04-28 01:21:38 +02:00
Matt Jankowski
9736753985 Fix remote follow (#2547)
* Add coverage for remote_follow#new

* Fix reference to authorize_follows/card partial
2017-04-28 00:16:44 +02:00
Eugen Rochko
2af4f3c4e2 Improve shared status verification (#2525)
* Instead of parsing shared status contents verbatim, make roundtrip
to purported original URL. Confirm that the "original" URL is from the
same domain as the author it claims to be from.

* Fix obvious typo, add comment

* Use URI look-up first

* Add test, update Goldfinger dependency to make less useless HTTP requests per Webfinger lookup
2017-04-27 17:06:47 +02:00
Matt Jankowski
b48f2cbc8b Catch error when server decryption fails on 2FA (#2512) 2017-04-27 15:18:21 +02:00
Eugen Rochko
88725d6ce8 OEmbed support for PreviewCard (#2337)
* OEmbed support for PreviewCard

* Improve ProviderDiscovery code failure treatment

* Do not crawl links if there is a content warning, since those
don't display a link card anyway

* Reset db schema

* Fresh migrate

* Fix rubocop style issues
Fix #1681 - return existing access token when applicable instead of creating new

* Fix test

* Extract http client to helper

* Improve oembed controller
2017-04-27 14:42:22 +02:00
Matt Jankowski
0618f09939 Add spec coverage and refactor authorize_follows controller (#2505) 2017-04-27 00:19:53 +02:00
Matt Jankowski
8857cabca4 Domain block service cleanup (#2490)
* Add coverage for domain block service with silence

* Get rid of warning about find_each and order

* Move domain_block to attr_reader

* Move optional clear_media into silence_accounts method

* Use blocked_domain method to reduce passed vars

* Extract blocked_domain_accounts method to find accounts on the domain

* Extract media_from_blocked_domain method to find relevant attachments

* Separate destruction of account images and account attachments
2017-04-26 20:09:01 +02:00
Matt Jankowski
affd75936e Add spec for Status.as_home_timeline (#2451) 2017-04-26 20:08:30 +02:00
Francis Chong
193dddb433 Add media dimensions (#2448)
* Fixes #1985

- add migration AddMediaAttachmentMeta, which add meta field to media_attachments
- before saving attachment, set file meta if needed
- add meta in api

* add spec

* align the “size” format for image and video

* fix code climate

* fixes media_attachment_spec.rb
2017-04-26 03:48:12 +02:00
Guillaume Lo Re
7177e37b99 Stricter whitelist rules (#2213)
* Stricter whitelist rules

* Linting

* Added spec for blacklisting

* Test subdomain blacklist on domain whitelist

* No need to split

* Change spec name
2017-04-26 01:22:51 +02:00
alpaca-tc
9317ec8eb1 Localize with i18n for Devise::FailureApp (#2309)
This PR fixes I18n.locale for rake middlewares. Mastodon uses Devise that depends on Warden.
Warden::Manager can be found in rake middleware. It is outside of the controller.

In the case of authentication failed, warden calls throw(:warden). At the time Warden::Manager
delegates request to failure_app to generate response and flash[:alert] after catching it.
Unfortunately, I18n.locale is already reset then because I18n.with_locale is enabled only
inside the controller. If we used I18n.locale=, Devise::FailureApp could get the current locale.
2017-04-25 15:06:41 +02:00
Evan Minto
122d59ac41 Change ActivityPub paging to match spec. Clean up ActivityPub outbox changes. (#2410)
* Change ActivityPub paging to match spec. Clean up ActivityPub outbox changes.

* Fix code style and test failures for OutboxController.

* Attempt to fix CI errors.
2017-04-25 15:06:06 +02:00
Eugen Rochko
8b5179d006 Fix #2402 - Add Idempotency-Key header to PostStatusService that prevents (#2419)
duplicates. Web UI regenerates UUID for that header every time the compose
form is changed or successfully submitted

Also, fix Farsi i18n overwriting the English one
2017-04-25 15:04:49 +02:00
Yamagishi Kazutoshi
3ea5b948a4 Decodes URL containing IDN (#2436) 2017-04-25 15:03:51 +02:00
178inaba
d2159deaf2 Optimize account search (#2421) 2017-04-25 04:44:43 +02:00
Eugen
17c591ffba Punycode URI normalization (#2370)
* Fix #2119 - Whenever about to send a HTTP request, normalize the URI

* Add test for IDN request in FetchLinkCardService

* Perform IDN normalization on domains before they are stored in the DB
2017-04-25 02:47:31 +02:00
Eugen
338df98ddf Add more FeedManager#filter? tests (#2413) 2017-04-24 22:37:24 +02:00
Matt Jankowski
092fdc89fa Fix flashes partial render error for controllers which don't inherit from application controller (#2400)
* Add failing spec for oauth/authorized_applications controller

* Use explicit reference to flashes partial from admin layout

Because some of the controllers which use the admin layout do not inherit from
application controller, this partial is not in their view path.
2017-04-24 17:30:30 +02:00
Eugen
501514960a Followers-only post federation (#2111)
* Make private toots get PuSHed to subscription URLs that belong to domains where you have approved followers

* Authorized followers controller, stub for bulk action

* Soft block in the background

* Add simple test for new controller

* Rename Settings::FollowersController to Settings::FollowerDomainsController, paginate results,
rename "private" post setting to "followers-only", fix pagination style, improve post privacy
preferences style, improve warning style

* Extract compose form warnings into own container, show warning when posting to followers-only with unlocked account
2017-04-24 00:38:37 +02:00
Matt Jankowski
948dd26931 Fix deprecation warnings on activitypub controller specs (#2357) 2017-04-23 19:10:04 +02:00
Matt Jankowski
55e1e12b7d Ensure i18n matches options (#2358)
* Add failing spec showing that human_locales does not match what i18n knows about

* Add missing `ar` key for arabic to human locales

* Remove duplicate `id` key from available locales

* Sort keys in human locales list

* Add spec for human_locale helper
2017-04-23 19:09:52 +02:00
Joël Quenneville
1cf9e14a41 Test embedded_view related code in a helper (#2282)
The two methods `StreamEntriesHelper#stream_link_target` and
`StreamEntriesHelper#acct` are based on checking whether we are running
in an embedded view.

This adds some test helper code to make the testing easier. We extracted
some "magic strings" to constants to lower the coupling in the specs.
2017-04-23 06:05:52 +02:00
Joël Quenneville
0c2fe22bc1 Add test coverage to CSS class generation (#2285)
The code that generates CSS is based on a lot of boolean conditions.
The possible combinations of these grows exponentially as we add more
conditions.

Since most of the code is conditional on a single boolean, we tested the
following:

1. All `false`
2. All `true`
3. Each individual flag set to `true`

The methods tested are:

* `StreamEntriesHelper#style_classes`
* `StreamEntriesHelper#microformats_classes`
* `StreamEntriesHelper#microformats_h_class`
2017-04-23 06:04:32 +02:00
Evan Minto
66fd8e7821 ActivityPub: Add basic, read-only support for Outboxes, Notes, and Create/Announce Activities (#2197)
* Clean up collapsible components

* Expose user Outboxes and AS2 representations of statuses

* Save work thus far.

* Fix bad merge.

* Save my work

* Clean up pagination.

* First test working.

* Add tests.

* Add Forbidden error template.

* Revert yarn.lock changes.

* Fix code style deviations and use localized instead of hardcoded English text.
2017-04-23 05:21:10 +02:00
Ash Furrow
723f25a999 Admin UI for confirming users (#2245)
* Shows confirmed status in list.

* Adds ability to confirm users in admin UI.

* Added new english translations.

* Addresses feedback from #2245.

* More feedback.
2017-04-23 04:43:42 +02:00
Wesley Ellis
a1174a6d7e Add og:url tags (#2298)
* add og:url to stream_item view

* add og:url and text to about/show

* add og:url to profile, followers and follows pages
2017-04-22 17:29:46 +02:00
Matt Jankowski
629d35e6f5 [WIP] Html lang on statuses (#2297)
* Add html lang attributes around statuses

* Remove urls from language detection
2017-04-22 04:26:25 +02:00
Matt Jankowski
67dea31b0f 2FA controller cleanup (#2296)
* Add spec coverage for settings/two_factor_auth area

* extract setup method for qr code

* Move otp required check to before action

* Merge method only used once

* Remove duplicate view

* Consolidate creation of @codes for backup

* Move settings/2fq#recovery_codes to settings/recovery_codes#create

* Rename settings/two_factor_auth#disable to #destroy

* Add coverage for the otp required path on 2fa#show

* Clean up the recovery codes list styles

* Move settings/two_factor_auth to settings/two_factor_authentication

* Reorganize the settings two factor auth area

Updated to use a flow like:

- settings/two_factor_authentication goes to a #show view which has a button
  either enable or disable 2fa on the account
- the disable button turns off the otp requirement for the user
- the enable button cycles the user secret and redirects to a confirmation page
- the confirmation page is a #new view which shows the QR code for user
- that page posts to #create which verifies the code, and creates the recovery
  codes
- that create action shares a view with a recovery codes controller which can be
  used separately to reset codes if needed
2017-04-22 04:23:17 +02:00
Joël Quenneville
974ac467de Add tests for StreamEntriesHelper#rtl? (#2286)
We used some random Arabic characters to test that the various RTL
conditions got triggered.
2017-04-22 00:13:37 +02:00
Matt Jankowski
ee0c897bba User settings mutation (#2270)
* Add user spec for settings, highlight global default mutation issue

* Fix mutation issue caused by settings/preferences spec
2017-04-21 18:07:17 +02:00
Matt Jankowski
2dda356e3f Clean up settings/preferences controller (#2237)
* Add missing fields group on preferences page

* Clean up settings/preferences controller

* Extract a UserSettingsDecorator
2017-04-21 03:26:52 +02:00
Matt Jankowski
1ada494bb2 Admin settings controller refactor, add specs, cleanup (#2225)
* Add render_views for admin/settings spec

* Add coverage for admin/settings#update

* Add coverage for admin/settings typecasting open_registrations setting

* Simplify how admin/settings finds the value for updating

* Rely on activerecord to not update a value that hasnt changed

* Add coverage for non-existent setting

* Use a constant for boolean settings
2017-04-20 17:18:09 +02:00
Marcin Cieślak
59a77923b3 Add static gif for upload testing (#2168) 2017-04-19 23:47:50 +02:00
Eugen
2e4afccd9d Fix #2108 - Fix gif uploads (#2171)
* Fix #2108 - Fix gif uploads
Add specs for media attachment gifv conversion

* Add ffmpeg to travis

* Make travis install ffmpeg, not libav

* Switch travis to trusty
2017-04-19 23:21:00 +02:00
Eugen
1d47910d3b Fix possibility of unrightful webfinger redirect (#2147)
* Fix possibility of unrightful webfinger redirect

* Add more tests for FollowRemoteAccountService
2017-04-19 17:28:35 +02:00
abcang
70891a99a9 Fix html escape characters in the URL (#2138)
* fix character escaping in URL

* add tests

* put a comma after the last item

* add HTML escape test
2017-04-19 14:52:18 +02:00
Matt Jankowski
8bac0350d1 Restful refactor of accounts/ routes (#2133)
* Add routing specs for accounts followers and following actions

* Use more restful route naming for public account follow pages

Moves two actions:
- accounts#followers to accounts/follower_accounts#index
- accounts#following to accounts/following_accounts#index

Adds routing spec to ensure prior URLs are preserved.
2017-04-19 13:52:37 +02:00
Eugen
96268e76e6 Fix wording on admin instances view, fix typos in language detector spec (#2106) 2017-04-18 23:39:51 +02:00
Eugen
6bd1019235 Add more tests for AtomSerializer (#2096) 2017-04-18 22:30:17 +02:00
Matt Jankowski
297c11dba2 Language detection refactor (#2099)
* Extract detect_language to separate class

* Use default locale, not just en

* Add spec to confirm that whatlanguage cant identify empty string

* Allow account locale to override default in language detector

* PostStatusService supplies an account to detect language
2017-04-18 22:20:12 +02:00
happycoloredbanana
0a7588282a Remove API authentication for public statuses (after review) (#1919) 2017-04-18 21:58:57 +02:00
Matt Jankowski
55e1503522 Instances list in admin (#2095)
* Add admin/instances index action

* Add link to instances admin page

* View lists instances

* Instances, grouped by domain, ordered by count

* Use Account.remote scope

* Extract method: Account.by_domain_accounts
2017-04-18 21:09:07 +02:00
Matt Jankowski
66d8f99a30 Admin reports with accounts (#2092)
* Add a ReportFilter class

* Add reports and targeted_reports relationships to Account

* Use ReportFilter from admin/reports controller

* Link to admin/reports filtered views from admin account show view

* Add indexes to reports.account_id and reports.target_account_id
2017-04-18 19:36:18 +02:00
Matt Jankowski
16d50f60d1 Remove unused methods (#1730)
* Remove unused method #set_counters_maps from api controller

* Remove unused method #set_account_counters_maps from api controller

* Remove unused method Account#followers_domains

* Remove unused User.prolific scope

* Add mastodon:users:admins task to list all admin emails

* Use interpolated query style in Account.triadic_closures

* Coverage for Account.triadic_closures
2017-04-18 01:21:55 +02:00
Eugen
f03fb6c0ae Add forgotten AtomSerializer spec (#2047) 2017-04-18 01:07:46 +02:00
Matt Jankowski
389f8f8249 Instance helper to replace site title helper (#2038)
* Move site title helper to instance helper (name change only)

* Remove newline in <title> tag

* Add site_hostname helper method to wrap up local_domain value

* Use site_hostname helper in places that need local_domain value
2017-04-18 00:16:32 +02:00
Matt Jankowski
4ed12cd89a Remove i18n_spec file (#2014)
These were being skipped with `xit`, but we also just added the `unused` task to
the CI server, and it will be caught there. That seems good enough for this,
instead of requiring a full keys check on every local spec suite run.
2017-04-17 20:02:29 +02:00
Matt Jankowski
cc1361c149 Media controller specs (#2022)
* Add spec for media controller

* Add MediaAttachment.attached scope

* Simplify methods in media controller
2017-04-17 20:02:00 +02:00
Matt Jankowski
21a767dcfa Improve handling of HTTP_ACCEPT for webfinger (#2008)
This change includes:

- Improve the spec coverage for incoming request to the webfinger action
- For requests without an accept header (ie, what a browser might look like),
  return a JSON response.
- For requests with an explicit format of xml or json, return that format.
- For requests using an accept header, return that format.

Also adds failing spec showing webfinger does not return xml, which covers the
issue described in: https://github.com/tootsuite/mastodon/issues/1983
2017-04-17 19:58:03 +02:00
Tomohiro Suwa
3399dd7a66 Fix nil query_username (#2013) 2017-04-17 19:57:02 +02:00
saturday06
d87ee1167e Assign user locale on signup (#1982) 2017-04-17 10:29:08 +02:00
Eugen
5d710b1139 Make file attachment on MediaAttachment optional (#1865)
Create MediaAttachment but without actual file download when domain is blocked with reject_media set to true
Clean up old media files when creating a new domain block with reject_media set to true
Return remote_url in media attachments API if local file is not present
Undo domain block action in admin UI
Ability to enable reject_media from admin UI
2017-04-16 12:51:30 +02:00
Matt Jankowski
3834e1e69b View spec fix (#1888)
* Add option to disable verify partial doubles

* Add show_landing_strip? helper method

* Use show_landing_strip? helper in accounts and stream entries views

* Fix naming in view specs
2017-04-16 03:40:33 +02:00
Matt Jankowski
6670e6d33f Add password reset for users from admin accounts area (#1841) 2017-04-15 16:44:59 +02:00
Matt Jankowski
3b8908c114 About page contact email (#1839)
* Correct site_contact_email typo

* Separate about more page into partials, add specs
2017-04-15 13:33:25 +02:00
Matt Jankowski
355965c17b Silence Devise deprecation warning about TestHelpers (#1840)
Devise changed their approach -
3f3ec236bb

This change silences a deprecation warning about `TestHelpers`
2017-04-15 13:30:57 +02:00
Patrick Figel
df4ff9a8e1 Add recovery code support for two-factor auth (#1773)
* Add recovery code support for two-factor auth

When users enable two-factor auth, the app now generates ten
single-use recovery codes. Users are encouraged to print the codes
and store them in a safe place.

The two-factor prompt during login now accepts both OTP codes and
recovery codes.

The two-factor settings UI allows users to regenerated lost
recovery codes. Users who have set up two-factor auth prior to
this feature being added can use it to generate recovery codes
for the first time.

Fixes #563 and fixes #987

* Set OTP_SECRET in test enviroment

* add missing .html to view file names
2017-04-15 13:26:03 +02:00
Matt Jankowski
40fd1de488 Account search service refactor (#1791)
* Begin coverage for account search service

* Coverage for hashtag query

* Coverage for calling local vs remote find based on domain presence

* Spec to check that exact matches are not duped

* Coverage of resolve option

* Coverage for account being provided

* Start to refactor account search service

* Isolate query username and domain methods

* Isolate exact_match method

* Extract methods for local and remote results

* Simplify local vs remote and account isoliation

* Extract methods for local and remote results

* Simplify de-dupe of exact match

* Simplify logic to check for non exact remotes

* Cache some methods

* Remove nil from exact_match from results array

* Return exact matches first

* Use find_remote even with no domain

Account.find_local is just an alias for Account.find_remote(user, nil) - so we
can not bother with the conditional here, and call find_remote directly.
2017-04-15 03:17:07 +02:00
ThibG
31f0bcf804 Refresh webfinger (#1323)
* Refresh local info for remote accounts when webfinger returns new values

It only refreshes account info if one of the URLs or the public-key changes,
in which cases it refreshes the full info, re-downloading the feeds from that
user.

Some special handling should probably be done when the public key changes,
but I have been unable to find any use for it in Mastodon yet.

* Re-fetch remote users we aren't subscribed to.

This might induce performance issues, we might want to only do that for users
we explicitly attempted to subscribe but failed to.

* Refactor changes

* Do not refresh existing remote account details more than once a day

* Avoid re-fetching webfinger info in tests unless otherwise specified
2017-04-15 03:16:05 +02:00
Matt Jankowski
09540192c9 Rename admin/accounts_helper to admin/filter_helper (#1816)
- Add some spec coverage for both methods
- Add explicit constant call-outs for where the params are from
2017-04-15 03:09:55 +02:00
Ben Roberts
89707ad0ac add basic microformats tests (#1803)
as suggested, moving to view tests rather than a controller test
replaces https://github.com/tootsuite/mastodon/pull/1786 which i will
close momentary
2017-04-15 02:37:00 +02:00
Chad Pytel
92cd207c50 Introduce capybara and first feature spec (#1801)
This commit introduces Capybara and the first feature spec.

I focused on coverage for log in for the first feature spec because that would
have prevented 624a9a7136 causing #1236.
2017-04-15 01:21:02 +02:00
Matt Jankowski
f5cd138323 Improve i18n chooser (#1804)
* Add locale spec with failing locale plus region check

* Use a more accurate locale when supplied by browser headers

Previously we were using a matching option which would use the first locale
available which matched the locale portion, even if a region was specified.

This changes to first try to find an exact match, and then fall back to the
region, and then fall back to the  default.

* Clean up default_locale method
2017-04-15 01:12:39 +02:00
Matt Jankowski
8b74aa4217 Admin reports controller improvements (#1714)
* Simplify admin/reports controller filtering for index

* Rename parameter to resolved

* Fix issue where reports view could not access filter_link_to

* Add coverage for admin/reports controller

* DRY up resolution of related reports for target account

* Clean up admin/reports routes

* Add Report#statuses method

* DRY up current account action taken params

* Rubocop styles
2017-04-14 11:10:28 +02:00
Matt Jankowski
b330d1f000 Organize coverage dirs (#1695)
* Add `Presenters` group to SimpleCov configuration

* Move validators to app/validators, add to simplecov config
2017-04-13 16:00:31 +02:00
Matt Jankowski
4fe5e04ea4 Fix csv export coverage in export spec (#1691) 2017-04-13 15:29:30 +02:00
Matt Jankowski
137100dcf3 Clean up well-known routes/controllers (#1649)
* Add request spec for host meta route returning xml

* Add routing spec for xrd routes

* Update well-known routes

* Move webfinger and host-meta actions to their own controllers
2017-04-13 13:09:07 +02:00
Matt Jankowski
3a9eb81a80 Admin accounts controller cleanup (#1664)
* Remove unused account_params method in admin/accounts controller

* Introduce AccountFilter to find accounts

* Use AccountFilter in admin/accounts controller

* Use more restful routes admin silence and suspension area

* Add admin/silences and admin/suspensions controllers
2017-04-13 13:04:23 +02:00
Matt Jankowski
0e39cc6a35 Settings export refactor (#1646)
* Refactor Export to take an account and know about the export types

* Use Export instance in settings/exports#show
2017-04-13 13:02:02 +02:00
Matt Jankowski
f24daa399b Remove pending specs for methods that dont exist (#1658)
* Remote spec for non-existent entry_classes helper method

This method no longer exists, and is handled by a local variable in a partial
instead.

* Remove spec for non-existent Account#ping! method
2017-04-13 02:51:13 +02:00
Eugen
5dc73339ae Fix tests issue introduced in #1607 (#1639) 2017-04-13 02:17:40 +02:00
Matt Jankowski
aa90798386 Webfinger resource to extract username from resource string (#1607)
* Add WebfingerResource class to extract usernames

* Use WebfingerResource in xrd#webfinger
2017-04-12 18:22:38 +02:00
Matt Jankowski
7f0a865b05 Allow import/export of mutes list (#1541)
* Allow export of mutes list

* Allow importing of mutes list

* Refactor to use Settings::Exports::BaseController and DRY up exports code
2017-04-12 18:20:44 +02:00
Henry Smith
08fce08217 Add unit tests for https://github.com/tootsuite/mastodon/pull/1574 (#1584) 2017-04-12 17:16:59 +02:00
Matt Jankowski
dd1ae3b109 Simplify the way the embed view is created (#1590)
* Add coverage for embedded status view

* Refactor embed view to eliminate @external_links variable
2017-04-12 16:12:42 +02:00
Matt Jankowski
b352a8e5d4 Default to json type for webfinger requests (#1583) 2017-04-12 16:03:37 +02:00
Matt Jankowski
b155e6ccf5 Fix issue with intermittent api/v1/notifications failure (#1606)
The spec was checking the activity_id of the activities held in notifications
within the controller.

Because the activities are different models, it is possible that they are
created with the same database IDs, and when they are this spec fails because an
activity which should not count as a match is counted as one.
2017-04-12 15:53:54 +02:00
Matt Jankowski
3ddd936b03 Refactor exports controller (#1567)
* Add basic coverage for settings/exports controller

* Remove unused @account variable from settings/exports controller

* Add coverage for download export actions

* Remove deprecated `render :text` in favor of `send_data` for csv downloads

* Add model to handle exports

* Use Export class in settings/exports controller

* Simplify settings/exports controller methods

* Move settings/export to more restful routes
2017-04-11 22:00:43 +02:00
Matt Jankowski
89e8e110c8 Imports controller errors (#1553)
* Add spec for settings/imports controller

* Add failing spec for settings/imports#create

* Fix broken imports

* Refactor ImportWorker
2017-04-11 21:40:14 +02:00
Matt Jankowski
e60286a344 Remove unused methods from StreamEntriesHelper (#1365)
Removes:

- avatar_for_status_url
- relative_time
- reblogged_by_me_class
- favourited_by_me_class
2017-04-11 19:32:02 +02:00
Matt Jankowski
53850bce93 Remove unused AtomBuilderHelper (#1364)
This was used in the views/atom/user_stream.xml.ruby file, which no longer is
used.
2017-04-11 19:31:22 +02:00
Matt Jankowski
4ada50985a Pagination improvements (#1445)
* Replace will_paginate with kaminari

* Use #page instead of #paginate in controllers

* Replace will_paginate.page_gap with pagination.truncate in i18n

* Customize kaminari views to match prior styles

* Set kaminari options to match prior behavior

* Replace will_paginate with paginate in views
2017-04-11 01:11:41 +02:00