* Use non-serial IDs
This change makes a number of nontrivial tweaks to the data model in
Mastodon:
* All IDs are now 8 byte integers (rather than mixed 4- and 8-byte)
* IDs are now assigned as:
* Top 6 bytes: millisecond-resolution time from epoch
* Bottom 2 bytes: serial (within the millisecond) sequence number
* See /lib/tasks/db.rake's `define_timestamp_id` for details, but
note that the purpose of these changes is to make it difficult to
determine the number of objects in a table from the ID of any
object.
* The Redis sorted set used for the feed will have values used to look
up toots, rather than scores. This is almost always the same as the
existing behavior, except in the case of boosted toots. This change
was made because Redis stores scores as double-precision floats,
which cannot store the new ID format exactly. Note that this doesn't
cause problems with sorting/pagination, because ZREVRANGEBYSCORE
sorts lexicographically when scores are tied. (This will still cause
sorting issues when the ID gains a new significant digit, but that's
extraordinarily uncommon.)
Note a couple of tradeoffs have been made in this commit:
* lib/tasks/db.rake is used to enforce many/most column constraints,
because this commit seems likely to take a while to bring upstream.
Enforcing a post-migrate hook is an easier way to maintain the code
in the interim.
* Boosted toots will appear in the timeline as many times as they have
been boosted. This is a tradeoff due to the way the feed is saved in
Redis at the moment, but will be handled by a future commit.
This would effectively close Mastodon's #1059, as it is a
snowflake-like system of generating IDs. However, given how involved
the changes were simply within Mastodon, it may have unexpected
interactions with some clients, if they store IDs as doubles
(or as 4-byte integers). This was a problem that Twitter ran into with
their "snowflake" transition, particularly in JavaScript clients that
treated IDs as JS integers, rather than strings. It therefore would be
useful to test these changes at least in the web interface and popular
clients before pushing them to all users.
* Fix JavaScript interface with long IDs
Somewhat predictably, the JS interface handled IDs as numbers, which in
JS are IEEE double-precision floats. This loses some precision when
working with numbers as large as those generated by the new ID scheme,
so we instead handle them here as strings. This is relatively simple,
and doesn't appear to have caused any problems, but should definitely
be tested more thoroughly than the built-in tests. Several days of use
appear to support this working properly.
BREAKING CHANGE:
The major(!) change here is that IDs are now returned as strings by the
REST endpoints, rather than as integers. In practice, relatively few
changes were required to make the existing JS UI work with this change,
but it will likely hit API clients pretty hard: it's an entirely
different type to consume. (The one API client I tested, Tusky, handles
this with no problems, however.)
Twitter ran into this issue when introducing Snowflake IDs, and decided
to instead introduce an `id_str` field in JSON responses. I have opted
to *not* do that, and instead force all IDs to 64-bit integers
represented by strings in one go. (I believe Twitter exacerbated their
problem by rolling out the changes three times: once for statuses, once
for DMs, and once for user IDs, as well as by leaving an integer ID
value in JSON. As they said, "If you’re using the `id` field with JSON
in a Javascript-related language, there is a very high likelihood that
the integers will be silently munged by Javascript interpreters. In most
cases, this will result in behavior such as being unable to load or
delete a specific direct message, because the ID you're sending to the
API is different than the actual identifier associated with the
message." [1]) However, given that this is a significant change for API
users, alternatives or a transition time may be appropriate.
1: https://blog.twitter.com/developer/en_us/a/2011/direct-messages-going-snowflake-on-sep-30-2011.html
* Restructure feed pushes/unpushes
This was necessary because the previous behavior used Redis zset scores
to identify statuses, but those are IEEE double-precision floats, so we
can't actually use them to identify all 64-bit IDs. However, it leaves
the code in a much better state for refactoring reblog handling /
coalescing.
Feed-management code has been consolidated in FeedManager, including:
* BatchedRemoveStatusService no longer directly manipulates feed zsets
* RemoveStatusService no longer directly manipulates feed zsets
* PrecomputeFeedService has moved its logic to FeedManager#populate_feed
(PrecomputeFeedService largely made lots of calls to FeedManager, but
didn't follow the normal adding-to-feed process.)
This has the effect of unifying all of the feed push/unpush logic in
FeedManager, making it much more tractable to update it in the future.
Due to some additional checks that must be made during, for example,
batch status removals, some Redis pipelining has been removed. It does
not appear that this should cause significantly increased load, but if
necessary, some optimizations are possible in batch cases. These were
omitted in the pursuit of simplicity, but a batch_push and batch_unpush
would be possible in the future.
Tests were added to verify that pushes happen under expected conditions,
and to verify reblog behavior (both on pushing and unpushing). In the
case of unpushing, this includes testing behavior that currently leads
to confusion such as Mastodon's #2817, but this codifies that the
behavior is currently expected.
* Rubocop fixes
I could swear I made these changes already, but I must have lost them
somewhere along the line.
* Address review comments
This addresses the first two comments from review of this feature:
https://github.com/tootsuite/mastodon/pull/4801#discussion_r139336735https://github.com/tootsuite/mastodon/pull/4801#discussion_r139336931
This adds an optional argument to FeedManager#key, the subtype of feed
key to generate. It also tests to ensure that FeedManager's settings are
such that reblogs won't be tracked forever.
* Hardcode IdToBigints migration columns
This addresses a comment during review:
https://github.com/tootsuite/mastodon/pull/4801#discussion_r139337452
This means we'll need to make sure that all _id columns going forward
are bigints, but that should happen automatically in most cases.
* Additional fixes for stringified IDs in JSON
These should be the last two. These were identified using eslint to try
to identify any plain casts to JavaScript numbers. (Some such casts are
legitimate, but these were not.)
Adding the following to .eslintrc.yml will identify casts to numbers:
~~~
no-restricted-syntax:
- warn
- selector: UnaryExpression[operator='+'] > :not(Literal)
message: Avoid the use of unary +
- selector: CallExpression[callee.name='Number']
message: Casting with Number() may coerce string IDs to numbers
~~~
The remaining three casts appear legitimate: two casts to array indices,
one in a server to turn an environment variable into a number.
* Only implement timestamp IDs for Status IDs
Per discussion in #4801, this is only being merged in for Status IDs at
this point. We do this in a migration, as there is no longer use for
a post-migration hook. We keep the initialization of the timestamp_id
function as a Rake task, as it is also needed after db:schema:load (as
db/schema.rb doesn't store Postgres functions).
* Change internal streaming payloads to stringified IDs as well
This is equivalent to 591a9af356faf2d5c7e66e3ec715502796c875cd from
#5019, with an extra change for the addition to FeedManager#unpush.
* Ensure we have a status_id_seq sequence
Apparently this is not a given when specifying a custom ID function,
so now we ensure it gets created. This uses the generic version of this
function to more easily support adding additional tables with timestamp
IDs in the future, although it would be possible to cut this down to a
less generic version if necessary. It is only run during db:schema:load
or the relevant migration, so the overhead is extraordinarily minimal.
* Transition reblogs to new Redis format
This provides a one-way migration to transition old Redis reblog entries
into the new format, with a separate tracking entry for reblogs.
It is not invertible because doing so could (if timestamp IDs are used)
require a database query for each status in each users' feed, which is
likely to be a significant toll on major instances.
* Address review comments from @akihikodaki
No functional changes.
* Additional review changes
* Heredoc cleanup
* Run db:schema:load hooks for test in development
This matches the behavior in Rails'
ActiveRecord::Tasks::DatabaseTasks.each_current_configuration, which
would otherwise break `rake db:setup` in development.
It also moves some functionality out to a library, which will be a good
place to put additional related functionality in the near future.
Additionally, ActivityPub::FetchRemoteStatusService no longer parses
activities.
OStatus::Activity::Creation no longer delegates to ActivityPub because
the provided ActivityPub representations are not signed while OStatus
representations are.
* Return sensible HTTP status for ActivityPub inbox processing
* Return sensible HTTP status for salmon slap processing
* Return additional information to debug signature verification failures
* Fix order of paginated accounts in FollowerDomainsController
Unordered pagination could result in unexpected behavior.
* Cover Settings::FollowerDomainsController more
If the signature could not be verified and the webfinger of the account
was last retrieved longer than the cache period, try re-resolving the
account and then attempting to verify the signature again
* Fix#117 - Add ability to specify alternative text for media attachments
- POST /api/v1/media accepts `description` straight away
- PUT /api/v1/media/:id to update `description` (only for unattached ones)
- Serialized as `name` of Document object in ActivityPub
- Uploads form adjusted for better performance and description input
* Add tests
* Change undo button blend mode to difference
* Add support for selecting a theme
* Fix codeclimate issues
* Look up site default style if current user is not available due to e.g. not being logged in
* Remove outdated comment in common.js
* Address requested changes in themes PR
* Fix codeclimate issues
* Explicitly check current_account in application controller and only check theme availability if non-nil
* codeclimate
* explicit precedence with &&
* Fix code style in application_controller according to @nightpool's suggestion, use default style in embedded.html.haml
* codeclimate: indentation + return
When a new user confirms their e-mail, bootstrap their home timeline
by automatically following a set of accounts. By default, all local
admin accounts (that are unlocked). Can be customized by new admin
setting (comma-separated usernames, local and unlocked only)
* Decouple Status#local? from uri being nil
* Replace on-the-fly URI generation with stored URIs
- Generate URI in after_save hook for local statuses
- Use static value in TagManager when available, fallback to tag format
- Make TagManager use ActivityPub::TagManager to understand new format
- Adjust tests
* Use other heuristic for locality of old statuses, do not perform long query
* Exclude tombstone stream entries from Atom feed
* Prevent nil statuses from landing in Pubsubhubbub::DistributionWorker
* Fix URI not being saved (#4818)
* Add more specs for Status
* Save generated uri immediately
and also fix method order to minimize diff.
* Fix alternate HTML URL in Atom
* Fix tests
* Remove not-null constraint from statuses migration to speed it up
* Make PreviewCard records reuseable between statuses
**Warning!** Migration truncates preview_cards tablec
* Allow a wider thumbnail for link preview, display it in horizontal layout (#4648)
* Delete preview cards files before truncating
* Rename old table instead of truncating it
* Add mastodon:maintenance:remove_deprecated_preview_cards
* Ignore deprecated_preview_cards in schema definition
* Fix null behaviour
- Use statuses controller for embeds instead of stream entries controller
- Prefer /@:username/:id/embed URL for embeds
- Use /@:username as author_url in OEmbed
- Add follow link to embeds which opens web intent in new window
- Use redis cache in development
- Cache entire embed
* Allow multiple pinned statuses to be shown and make them be ordered by pinned date
* Set timestamps NOT NULL
* Make single-line pinned_statuses
* Spec for pinned_statuses
* Remove redundant empty line
* Process Create / Announce activity in FetchRemoteStatusService
* Use activity URL in ActivityPub for reblogs
* Redirect to the original status on StatusesController#show
* Add code for creating/managing apps to settings section
* Add specs for app changes
* Fix controller spec
* Fix view file I pasted over by mistake
* Add locale strings. Add 'my apps' to nav
* Add Client ID/Secret to App page. Add some visual separation
* Fix rubocop warnings
* Fix embarrassing typo
I lost an `end` statement while fixing a merge conflict.
* Add code for creating/managing apps to settings section
- Add specs for app changes
- Add locale strings. Add 'my apps' to nav
- Add Client ID/Secret to App page. Add some visual separation
- Fix some bugs/warnings
* Update to match code standards
* Trigger notification
* Add warning about not sharing API secrets
* Tweak spec a bit
* Cleanup fixture creation by using let!
* Remove unused key
* Add foreign key for application<->user
- Tries to avoid performing HTTP request if the keyId is an actor URI
- Likewise if the URI is a fragment URI on top of actor URI
- Resolves public key, returns owner if the owner links back to the key
* ActivityPub migration procedure
Once one account is detected as going from OStatus to ActivityPub,
invalidate WebFinger cache for other accounts from the same domain
* Unsubscribe from PuSH updates once we receive an ActivityPub payload
* Re-subscribe to PuSH unless already unsubscribed, regardless of protocol
* Add ActivityPub inbox
* Handle ActivityPub deletes
* Handle ActivityPub creates
* Handle ActivityPub announces
* Stubs for handling all activities that need to be handled
* Add ActivityPub actor resolving
* Handle conversation URI passing in ActivityPub
* Handle content language in ActivityPub
* Send accept header when fetching actor, handle JSON parse errors
* Test for ActivityPub::FetchRemoteAccountService
* Handle public key and icon/image when embedded/as array/as resolvable URI
* Implement ActivityPub::FetchRemoteStatusService
* Add stubs for more interactions
* Undo activities implemented
* Handle out of order activities
* Hook up ActivityPub to ResolveRemoteAccountService, handle
Update Account activities
* Add fragment IDs to all transient activity serializers
* Add tests and fixes
* Add stubs for missing tests
* Add more tests
* Add more tests
This PR adds a new notification cleaning mode, super perfectly tuned for accessibility, and removes the previous notification cleaning functionality as it's now redundant.
* w.i.p. notif clearing mode
* Better CSS for selected notification and shorter text if Stretch is off
* wip for rebase ~
* all working in notif clearing mode, except the actual removal
* bulk delete route for piggo
* cleaning + refactor. endpoint gives 422 for some reason
* formatting
* use the right route
* fix broken destroy_multiple
* load more notifs after succ cleaning
* satisfy eslint
* Removed CSS for the old notif delete button
* Tabindex=0 is mandatory
In order to make it possible to tab to this element you must have tab index = 0. Removing this violates WCAG and makes it impossible to use the interface without good eyesight and a mouse. So nobody with certain mobility impairments, vision impairments, or brain injuries would be able to use this feature if you don't have tabindex=0
* Corrected aria-label
Previous label implied a different behavior from what actually happens
* aria role localization & made the overlay behave like a checkbox
* checkboxes css and better contrast
* color tuning for the notif overlay
* fanceh checkboxes etc and nice backgrounds
* SHUT UP TRAVIS
* Do not raise unretryable exceptions in ResolveRemoteAccountService
* Removed fatal exceptions from ResolveRemoteAccountService
Exceptions that cannot be retried should not be raised. New exception
class for those that can be retried (Mastodon::UnexpectedResponseError)
* Wrap methods of ProcessFeedService::ProcessEntry in classes
This is a change same with 425acecfdb, except
that it has the following changes:
* Revert irrelevant change in find_or_create_conversation
* Fix error handling for RemoteActivity
* Introduce Ostatus name space
* Improve webfinger templates and make tests more flexible
* Clean up AS2 representation of actor
* Refactor outbox
* Create activities representation
* Add representations of followers/following collections, do not redirect /users/:username route if format is empty
* Remove unused translations
* ActivityPub endpoint for single statuses, add ActivityPub::TagManager for better
URL/URI generation
* Add ActivityPub::TagManager#to
* Represent all attachments as Document instead of Image/Video specifically
(Because for remote ones we may not know for sure)
Add mentions and hashtags representation to AP notes
* Add AP-resolvable hashtag URIs
* Use ActiveModelSerializers for ActivityPub
* Clean up unused translations
* Separate route for object and activity
* Adjust cc/to matrices
* Add to/cc to activities, ensure announce activity embeds target status and
not the wrapper status, add "id" to all collections
* Add Request class with HTTP signature generator
Spec: https://tools.ietf.org/html/draft-cavage-http-signatures-06
* Add HTTP signature verification concern
* Add test for SignatureVerification concern
* Add basic test for Request class
* Make PuSH subscribe/unsubscribe requests use new Request class
Accidentally fix lease_seconds not being set and sent properly, and
change the new minimum subscription duration to 1 day
* Make all PuSH workers use new Request class
* Make Salmon sender use new Request class
* Make FetchLinkService use new Request class
* Make FetchAtomService use the new Request class
* Make Remotable use the new Request class
* Make ResolveRemoteAccountService use the new Request class
* Add more tests
* Allow +-30 seconds window for signed request to remain valid
* Disable time window validation for signed requests, restore 7 days
as PuSH subscription duration (which was previous default due to a bug)
* Redesign the landing page, mount public timeline on it
* Adjust the standalone mounted component to the lacking of router
* Adjust auth layout pages to new design
* Fix tests
* Standalone public timeline polling every 5 seconds
* Remove now obsolete translations
* Add responsive design for new landing page
* Address reviews
* Add floating clouds behind frontpage form
* Use access token from public page when available
* Fix mentions and hashtags links, cursor on status content in standalone mode
* Add footer link to source code
* Fix errors on pages that don't embed the component, use classnames
* Fix tests
* Change anonymous autoPlayGif default to false
* When gif autoplay is disabled, hover to play
* Add option to hide the timeline preview
* Slightly improve alt layout
* Add elephant friend to new frontpage
* Display "back to mastodon" in place of "login" when logged in on frontpage
* Change polling time to 3s
* Added a success page to remote following
Includes follow-through links to web (the old redirect target) and back to the remote user's profile
* Use Account.new in spec instead of a fake with only id
(fixes spec)
* Fabricate(:account) over Account.new
* Remove self from the success text
(and all HTML with it)
* add a system_font_ui setting on the server
* Plug the system_font_ui on the front-end
* add EN/FR locales for the new setting
* put Roboto after all other fonts
* remove trailing whitespace so CodeClimate is happy
* fix user_spec.rb
* correctly write user_spect this time
* slightly better way of adding the classes
* add comments to the system-font stack for clarification
* use .system-font for the class instead
* don't use multiple lines for comments
* remove trailing whitespace
* use the classnames module for consistency
* use `mastodon-font-sans-serif` instead of Roboto directly
* Fix#3910 - Require OTP authentication to disable 2FA. Also, remove ability
to generate new OTP backup codes *after* initial backup codes were handed
out during activation
* Restore recovery code re-generation
* Improve display of some 2FA elements
* Add overview of active sessions
* Better display of browser/platform name
* Improve how browser information is stored and displayed for sessions overview
* Fix test
* Fix#2347 - Bind web UI access token to session
When you logout, session also destroys the access token, so it's no longer
valid. If access token is destroyed some other way, the session is also
destroyed, requiring a re-login.
Fix#1681 - Add scheduler to remove revoked access tokens and grants
* Fix test
* Add overview of active sessions
* Better display of browser/platform name
* Improve how browser information is stored and displayed for sessions overview
* Fix test
* Add form for account deletion
* If avatar or header are gone from source, remove them
* Add option to have SuspendAccountService remove user record, add tests
* Exclude suspended accounts from search
* Fix#2619 - When redis feed is empty, fall back to database
* Use redis value to return feed from database only while RegenerationWorker
hasn't finished running
* Fix specs
* Replace usage of reject!
When case insensitivity is enabled via devise's `config.case_insensitive_keys` then `.find_for_authentication` method needs to be used instead of `.find_by` because second mentioned returns `nil` when valid email with different cases is passed.
More info https://github.com/plataformatec/devise/wiki/How-To:-Use-case-insensitive-emails
* Improve default language decision
This change allows to takes account of accepted language determined by
the user agent even if the custom default locale of the instance is
configured.
* Cover Localized more
* Fix code style
Each of mute, favourite, reblog has been updated to:
- Have a separate controller with just a create and destroy action
- Preserve historical route names to not break the API
- Mild refactoring to break up long methods
* Add specs for api statuses routes
* Update favourited_by and reblogged_by api routes
* Move methods into new controllers
* Use load_accounts methods to simplify index actions
* Clean up load_accounts methods
* Clean up link header generation
* Check for link headers in specs
* Remove unused actions from api/v1/statuses controller
* Remove specs for moved actions
* Move ApiController to Api/BaseController
* API controllers inherit from Api::BaseController
* Add coverage for various error cases in api/base controller
* Coverage for rate limit headers
* Move rate limit headers methods to concern
* Move throttle check to condition on before_action
* Move match_data variable into method
* Move utc timestamp to separate method
* Move header setting into smaller methods
* specs cleanup
* Redirect to streaming_api_base_url
When Rails receives a request to streaming API, it most likely
means that there is another host which is configured to respond
to it. This is to redirect clients to that host if
`STREAMING_API_BASE_URL` is set as another host.
* Use the new Ruby 1.9 hash syntax
* Set delete_modal preference to true by default
* Does not show confirmation modal if delete_modal is false
* Add ja translation for preference setting page
In single user mode, visitors are redirected to the single user's
profile page. So, if you are the owner without a session, you start
from that page, click the login button and authenticate yourself
expecting you'll soon get started with the home page, but in reality
you'll get redirected back to where you started from -- your own
profile page.
This fixes the behavior by redirecting you home after login if you
have started from your own profile page.
This will reduce requests on who have only few statuses.
- Use next link header to detect more items from first request
- Omit next link header if result items are fewer than requested count
(It had omit it only if result was empty before)
Link headers in following/followers API should include follow_id as max_id/since_id.
However, these API use current_user's account_id instead of follow_id from #3167.
This causes irrelevant result on loading more users.
- Increase coverage to exercise all parts of each action
- Move into namespace to share common code
- Misc refactor of each action for smaller methods, simpler code
* Introduce recent scope to Status and StreamEntry
Introduce recent scope to Status and StreamEntry as Account has.
* Cover AccountsController more in AccountsController
* Add buttons to block and unblock domain
* Relationship API now returns "domain_blocking" status for accounts,
rename "block entire domain" to "hide entire domain", fix unblocking domain,
do not block notifications from domain-blocked-but-followed people, do
not send Salmons to domain blocked users
* Add test
* Personal domain blocks shouldn't affect Salmon after all, since in this
direction of communication the control is very thin when it comes to
public stuff. Best stay consistent and not affect federation in this way
* Ignore followers and follow request from domain blocked folks,
ensure account domain blocks are not created for empty domain,
and avoid duplicates in validation
* Purge followers when blocking domain (without soft-blocks, since they
are useless here)
* Add tests, fix local timeline being empty when having any domain blocks
* Add <ostatus:conversation /> tag to Atom input/output
Only uses ref attribute (not href) because href would be
the alternate link that's always included also.
Creates new conversation for every non-reply status. Carries
over conversation for every reply. Keeps remote URIs verbatim,
generates local URIs on the fly like the rest of them.
* Conversation muting - prevents notifications that reference a conversation
(including replies, favourites, reblogs) from being created. API endpoints
/api/v1/statuses/:id/mute and /api/v1/statuses/:id/unmute
Currently no way to tell when a status/conversation is muted, so the web UI
only has a "disable notifications" button, doesn't work as a toggle
* Display "Dismiss notifications" on all statuses in notifications column, not just own
* Add "muted" as a boolean attribute on statuses JSON
For now always false on contained reblogs, since it's only relevant for
statuses returned from the notifications endpoint, which are not nested
Remove "Disable notifications" from detailed status view, since it's
only relevant in the notifications column
* Up max class length
* Remove pending test for conversation mute
* Add tests, clean up
* Rename to "mute conversation" and "unmute conversation"
* Raise validation error when trying to mute/unmute status without conversation
* Adding account domain blocks that filter notifications and public timelines
* Add tests for domain blocks in notifications, public timelines
Filter reblogs of blocked domains from home
* Add API for listing and creating account domain blocks
* API for creating/deleting domain blocks, tests for Status#ancestors
and Status#descendants, filter domain blocks from them
* Filter domains in streaming API
* Update account_domain_block_spec.rb
* Add <ostatus:conversation /> tag to Atom input/output
Only uses ref attribute (not href) because href would be
the alternate link that's always included also.
Creates new conversation for every non-reply status. Carries
over conversation for every reply. Keeps remote URIs verbatim,
generates local URIs on the fly like the rest of them.
* Conversation muting - prevents notifications that reference a conversation
(including replies, favourites, reblogs) from being created. API endpoints
/api/v1/statuses/:id/mute and /api/v1/statuses/:id/unmute
Currently no way to tell when a status/conversation is muted, so the web UI
only has a "disable notifications" button, doesn't work as a toggle
* Display "Dismiss notifications" on all statuses in notifications column, not just own
* Add "muted" as a boolean attribute on statuses JSON
For now always false on contained reblogs, since it's only relevant for
statuses returned from the notifications endpoint, which are not nested
Remove "Disable notifications" from detailed status view, since it's
only relevant in the notifications column
* Up max class length
* Remove pending test for conversation mute
* Add tests, clean up
* Rename to "mute conversation" and "unmute conversation"
* Raise validation error when trying to mute/unmute status without conversation
Fix#2196 - Respond with 201 when Salmon accepted, 400 when unverified
Fix#2629 - Correctly handle confirm_domain? for local accounts
Unify rules for extracting author acct from XML, prefer <email>, fall back
to <name> + <uri> (see also #2017, #2172)
* Add coverage for create with empty acct value
* Add coverage for create with webfinger failure
* Add coverage for create with webfinger providing bad values
* Add coverage for create when webfinger is good
* Add coverage for session[:remote_follow] having data
* Simplify how remote follow pulls acct from session
* Remote follow behaves more like model
* Move the discovery portions of remote follow out of controller
* Check for suspended accounts
* Naive approached to timeline filtering
* Convert allowed_languages into a db column
* Allow users to choose languages to see statuses in
* Style list items as two columns
* Add a hint to explain language filtering preference
* Add coverage for admin/confirmations controller
* Coverage for statuses controller show action
* Add coverage for admin/domain_blocks controller
* Add coverage for settings/profiles#update
* OEmbed support for PreviewCard
* Improve ProviderDiscovery code failure treatment
* Do not crawl links if there is a content warning, since those
don't display a link card anyway
* Reset db schema
* Fresh migrate
* Fix rubocop style issues
Fix#1681 - return existing access token when applicable instead of creating new
* Fix test
* Extract http client to helper
* Improve oembed controller
This PR fixes I18n.locale for rake middlewares. Mastodon uses Devise that depends on Warden.
Warden::Manager can be found in rake middleware. It is outside of the controller.
In the case of authentication failed, warden calls throw(:warden). At the time Warden::Manager
delegates request to failure_app to generate response and flash[:alert] after catching it.
Unfortunately, I18n.locale is already reset then because I18n.with_locale is enabled only
inside the controller. If we used I18n.locale=, Devise::FailureApp could get the current locale.
* Change ActivityPub paging to match spec. Clean up ActivityPub outbox changes.
* Fix code style and test failures for OutboxController.
* Attempt to fix CI errors.
duplicates. Web UI regenerates UUID for that header every time the compose
form is changed or successfully submitted
Also, fix Farsi i18n overwriting the English one
* Fix#2119 - Whenever about to send a HTTP request, normalize the URI
* Add test for IDN request in FetchLinkCardService
* Perform IDN normalization on domains before they are stored in the DB
* Make private toots get PuSHed to subscription URLs that belong to domains where you have approved followers
* Authorized followers controller, stub for bulk action
* Soft block in the background
* Add simple test for new controller
* Rename Settings::FollowersController to Settings::FollowerDomainsController, paginate results,
rename "private" post setting to "followers-only", fix pagination style, improve post privacy
preferences style, improve warning style
* Extract compose form warnings into own container, show warning when posting to followers-only with unlocked account
* Clean up collapsible components
* Expose user Outboxes and AS2 representations of statuses
* Save work thus far.
* Fix bad merge.
* Save my work
* Clean up pagination.
* First test working.
* Add tests.
* Add Forbidden error template.
* Revert yarn.lock changes.
* Fix code style deviations and use localized instead of hardcoded English text.
* Shows confirmed status in list.
* Adds ability to confirm users in admin UI.
* Added new english translations.
* Addresses feedback from #2245.
* More feedback.
The force_ssl method from controllers does not add all of the options that the
sitewide configuration in a config block does. For example, HSTS enforcement is
not added by the controller method, but is added by this style.
* Add spec coverage for settings/two_factor_auth area
* extract setup method for qr code
* Move otp required check to before action
* Merge method only used once
* Remove duplicate view
* Consolidate creation of @codes for backup
* Move settings/2fq#recovery_codes to settings/recovery_codes#create
* Rename settings/two_factor_auth#disable to #destroy
* Add coverage for the otp required path on 2fa#show
* Clean up the recovery codes list styles
* Move settings/two_factor_auth to settings/two_factor_authentication
* Reorganize the settings two factor auth area
Updated to use a flow like:
- settings/two_factor_authentication goes to a #show view which has a button
either enable or disable 2fa on the account
- the disable button turns off the otp requirement for the user
- the enable button cycles the user secret and redirects to a confirmation page
- the confirmation page is a #new view which shows the QR code for user
- that page posts to #create which verifies the code, and creates the recovery
codes
- that create action shares a view with a recovery codes controller which can be
used separately to reset codes if needed
* Add render_views for admin/settings spec
* Add coverage for admin/settings#update
* Add coverage for admin/settings typecasting open_registrations setting
* Simplify how admin/settings finds the value for updating
* Rely on activerecord to not update a value that hasnt changed
* Add coverage for non-existent setting
* Use a constant for boolean settings
* Add routing specs for accounts followers and following actions
* Use more restful route naming for public account follow pages
Moves two actions:
- accounts#followers to accounts/follower_accounts#index
- accounts#following to accounts/following_accounts#index
Adds routing spec to ensure prior URLs are preserved.
* Add a ReportFilter class
* Add reports and targeted_reports relationships to Account
* Use ReportFilter from admin/reports controller
* Link to admin/reports filtered views from admin account show view
* Add indexes to reports.account_id and reports.target_account_id
* Remove unused method #set_counters_maps from api controller
* Remove unused method #set_account_counters_maps from api controller
* Remove unused method Account#followers_domains
* Remove unused User.prolific scope
* Add mastodon:users:admins task to list all admin emails
* Use interpolated query style in Account.triadic_closures
* Coverage for Account.triadic_closures
This change includes:
- Improve the spec coverage for incoming request to the webfinger action
- For requests without an accept header (ie, what a browser might look like),
return a JSON response.
- For requests with an explicit format of xml or json, return that format.
- For requests using an accept header, return that format.
Also adds failing spec showing webfinger does not return xml, which covers the
issue described in: https://github.com/tootsuite/mastodon/issues/1983
This introduces a new per-user preference called
"Auto-play animated GIFs", which is enabled by default. When a
user disables this setting, gifs in toots become click-to-play.
Previews of animated gifs were changed to display the video play
button so that users can distinguish them from regular images.
This setting also affects account avatars in the detailed account
view, which was changed to use the same hover-to-play mechanism
that is used for animated avatars in timelines.
Fixes#1652
* Basic onboarding modal that's shown to users once
* Lay out pages 2 through 5, add images, style modals (#1509)
* Lay out pages 2 through 5
Added images and laid out pages 2 through 5 in the jsx file. SCSS will
come, still working on just seeing if this works at all.
* Fix jsx errors, add images to modal pages, style modal pages
* Add animations to onboarding pager changes, improve wording and styling
* Finishing touches on the onboarding
* Add missing propTypes
* Update wording
* Rename admin.domain_block to admin.domain_blocks in prep for i18n improvement
* Use implicit controller/action path for i18n in admin/domain_blocks
* Add DomainBlock#accounts has_many
* Avoid i18n health warning for `en` locale by using symbol scope with :count
* Remove unused i18n key: plaintext_secret_html
* Remove unused i18n key two_factor_auth.warning
* Remove final will_paginate i18n keys
* Remove unused key two_factor_auth.recovery_codes
* Remove unused key: admin.reports.comment.none
* Remove unused reports. i18n namespace (moved to admin.reports)
* Ignore keys from locales which override activemodel and activerecord errors
* Revert "Remove unused key: admin.reports.comment.none"
This reverts commit 350ef2685fadc069e619bb6d1066190de195d942.
* Update i18n key reference to match moved location
* Add missing `en` keys to i18n
* Tell i18n-tasks to ignore missing attributes that dont need overwriting
* Add i18n-tasks unused to travis
Create MediaAttachment but without actual file download when domain is blocked with reject_media set to true
Clean up old media files when creating a new domain block with reject_media set to true
Return remote_url in media attachments API if local file is not present
Undo domain block action in admin UI
Ability to enable reject_media from admin UI
An attempt to open a brand new Mastodon instance configured
as SINGLE_USER_MODE=true will cause an exception.
Enable temporary registration if we have no users in the database
Fixes#1817
* Add recovery code support for two-factor auth
When users enable two-factor auth, the app now generates ten
single-use recovery codes. Users are encouraged to print the codes
and store them in a safe place.
The two-factor prompt during login now accepts both OTP codes and
recovery codes.
The two-factor settings UI allows users to regenerated lost
recovery codes. Users who have set up two-factor auth prior to
this feature being added can use it to generate recovery codes
for the first time.
Fixes#563 and fixes#987
* Set OTP_SECRET in test enviroment
* add missing .html to view file names
* Allow running mastodon on a different domain as the one used for identifying users
* Alter documentation of WEB_DOMAIN to make clear it shouldn't be used unless the admin knows what they are doing
* Compare to web_domain instead of local_domain when dealing with feeds/API
* Correctly identify mentions to local accounts
Mentions URLs point to the person's web profile, i.e., the user page served on WEB_DOMAIN.
* Add locale spec with failing locale plus region check
* Use a more accurate locale when supplied by browser headers
Previously we were using a matching option which would use the first locale
available which matched the locale portion, even if a region was specified.
This changes to first try to find an exact match, and then fall back to the
region, and then fall back to the default.
* Clean up default_locale method
* Simplify admin/reports controller filtering for index
* Rename parameter to resolved
* Fix issue where reports view could not access filter_link_to
* Add coverage for admin/reports controller
* DRY up resolution of related reports for target account
* Clean up admin/reports routes
* Add Report#statuses method
* DRY up current account action taken params
* Rubocop styles
* Add request spec for host meta route returning xml
* Add routing spec for xrd routes
* Update well-known routes
* Move webfinger and host-meta actions to their own controllers
* Remove unused account_params method in admin/accounts controller
* Introduce AccountFilter to find accounts
* Use AccountFilter in admin/accounts controller
* Use more restful routes admin silence and suspension area
* Add admin/silences and admin/suspensions controllers
* Add basic coverage for settings/exports controller
* Remove unused @account variable from settings/exports controller
* Add coverage for download export actions
* Remove deprecated `render :text` in favor of `send_data` for csv downloads
* Add model to handle exports
* Use Export class in settings/exports controller
* Simplify settings/exports controller methods
* Move settings/export to more restful routes
* Replace will_paginate with kaminari
* Use #page instead of #paginate in controllers
* Replace will_paginate.page_gap with pagination.truncate in i18n
* Customize kaminari views to match prior styles
* Set kaminari options to match prior behavior
* Replace will_paginate with paginate in views
* Add Admin::BaseController to wrap admin area
Extracts the setting of the `admin` layout and verifying that users are admins
to a common base class for the admin/ controllers.
* Add basic coverage for admin/reports and admin/settings controllers
Sign-in redirects you back to last visited URL, but in case of API requests,
this sometimes redirected users to an API URL that, of course, greeted them
with an {"error":"The access token is invalid"}
* Use HTTP Accept-Language to detect locale
* Fix gem order to comply with codeclimate
* Sort gem to comply with rubocop
* I18n.default_locale fallback when there is no accept-language header
* Allow users to update their Account in the API
It would be nice for API clients to be able to allow users to update
their accounts without having to wrap Mastodon in a web view. This patch
adds an API endpoint to let users submit a PATCH for their account.
Signed-off-by: David Celis <me@davidcel.is>
* Add /api/v1/accounts/update_credentials to the API docs
Signed-off-by: David Celis <me@davidcel.is>
* Make /api/v1/timelines/public and /api/v1/timelines/tag/:id public
Fix#1156 - respect query params when generating pagination links in API
* Apply pagination fix to more APIs
Setting of locale in controller extracted to Localized concern,
the doorkeeper authorized applications controller moved under
custom namespace with inclusion of Localized, which resolves the
"it sometimes appears in a different random language" bug
* Rewrite Atom generation from stream entries to use Ox instead of Nokogiri::Builder
StreamEntry is now limited to only statuses, which allows some optimization. Removed
extra queries on AccountsController#show. AtomSerializer instead of AtomBuilderHelper
used in AccountsController#show, StreamEntriesController#show, StreamEntryRenderer
and PubSubHubbub::DistributionWorker
PubSubHubbub::DistributionWorker moves n+1 DomainBlock query to PubSubHubbub::DeliveryWorker
instead.
All Salmon slaps that aren't based on StreamEntry still use AtomBuilderHelper and Nokogiri
* All Salmon slaps now use Ox instead of Nokogiri. No touch from status on account
Modified uploadCompose action to send media ids of attached
media when sending a request. Modified create method in MediaController
to check if when posting a video, there are no other media attached
to the status by looking at the media ids sent from the uploadCompose
action.
to only those from local users; Add link to "extended information" to
getting started in the UI; Add defaults for posting privacy; Change
how publish button looks depending on posting privacy chosen
Federate spoiler_text using warning attribute on <content /> instead of a <category term="spoiler" />
Clean up schema file from accidental development migrations
Filters out hidden stream entries from Atom feed
Blocks now generate hidden stream entries, can be used to federate blocks
Private statuses cannot be reblogged (generates generic 422 error for now)
POST /api/v1/statuses now takes visibility=(public|unlisted|private) param instead of unlisted boolean
Statuses JSON now contains visibility=(public|unlisted|private) field
resources that require a user context vs those that don't (such as public timeline)
/api/v1/statuses/public -> /api/v1/timelines/public
/api/v1/statuses/home -> /api/v1/timelines/home
/api/v1/statuses/mentions -> /api/v1/timelines/mentions
/api/v1/statuses/tag/:tag -> /api/v1/timelines/tag/:tag
This is a big one, so let me enumerate:
Accounts as well as stream entry pages now contain Link headers that
reference the Atom feed and Webfinger URL for the former and Atom entry
for the latter. So you only need to HEAD those resources to get that
information, no need to download and parse HTML <link>s.
ProcessFeedService will now queue ThreadResolveWorker for each remote
status that it cannot find otherwise. Furthermore, entries are now
processed in reverse order (from bottom to top) in case a newer entry
references a chronologically previous one.
ThreadResolveWorker uses FetchRemoteStatusService to obtain a status
and attach the child status it was queued for to it.
FetchRemoteStatusService looks up the URL, first with a HEAD, tests
if it's an Atom feed, in which case it processes it directly. Next
for Link headers to the Atom feed, in which case that is fetched
and processed. Lastly if it's HTML, it is checked for <link>s to the Atom
feed, and if such is found, that is fetched and processed. The account for
the status is derived from author/name attribute in the XML and the hostname
in the URL (domain). FollowRemoteAccountService and ProcessFeedService
are used.
This means that potentially threads are resolved recursively until a dead-end
is encountered, however it is performed asynchronously over background jobs,
so it should be ok.
POST /api/statuses Params: status (text contents), in_reply_to_id (optional)
GET /api/statuses/:id
POST /api/statuses/:id/reblog
GET /api/accounts/:id
GET /api/accounts/:id/following
GET /api/accounts/:id/followers
POST /api/accounts/:id/follow
POST /api/accounts/:id/unfollow
POST /api/follows Params: uri (e.g. user@domain)
OAuth authentication is currently disabled, but the API can be used with HTTP Auth.
