Commit Graph

28 Commits

Author SHA1 Message Date
David Yip 1ab12ba38e Merge remote-tracking branch 'origin/master' into merge-upstream 2017-11-17 17:40:00 -06:00
Eugen Rochko 24cafd73a2
Lists (#5703)
* Add structure for lists

* Add list timeline streaming API

* Add list APIs, bind list-account relation to follow relation

* Add API for adding/removing accounts from lists

* Add pagination to lists API

* Add pagination to list accounts API

* Adjust scopes for new APIs

- Creating and modifying lists merely requires "write" scope
- Fetching information about lists merely requires "read" scope

* Add test for wrong user context on list timeline

* Clean up tests
2017-11-18 00:16:48 +01:00
Surinna Curtis 35fbdc36f9 Merge tootsuite/master at 3023725936 2017-11-16 01:21:16 -06:00
Daniel Hunsaker cf7e840990 Update model annotations to use BIGINT for IDs (#5461)
All the migrations have been updated to use BIGINTs for ID fields in the DB, but ActiveRecord needs to be told to treat those values as BIGINT as well. This PR does that.
2017-11-12 16:18:50 +01:00
aschmitz 4944515020 "Show reblogs" per-follower UI/database changes
TODO:

* Tests (particularly for FollowRequests).
* Anything to respect the setting when putting reblogs in timelines.
2017-11-09 08:41:10 -06:00
aschmitz 97c02c3389 Make IdsToBigints (mostly!) non-blocking (#5088)
* Make IdsToBigints (mostly!) non-blocking

This pulls in GitLab's MigrationHelpers, which include code to make
column changes in ways that Postgres can do without locking. In general,
this involves creating a new column, adding an index and any foreign
keys as appropriate, adding a trigger to keep it populated alongside
the old column, and then progressively copying data over to the new
column, before removing the old column and replacing it with the new
one.

A few changes to GitLab's MigrationHelpers were necessary:

* Some changes were made to remove dependencies on other GitLab code.
* We explicitly wait for index creation before forging ahead on column
  replacements.
* We use different temporary column names, to avoid running into index
  name length limits.
* We rename the generated indices back to what they "should" be after
  replacing columns.
* We rename the generated foreign keys to use the new column names when
  we had to create them. (This allows the migration to be rolled back
  without incident.)

# Big Scary Warning

There are two things here that may trip up large instances:

1. The change for tables' "id" columns is not concurrent. In
   particular, the stream_entries table may be big, and does not
   concurrently migrate its id column. (On the other hand, x_id type
   columns are all concurrent.)
2. This migration will take a long time to run, *but it should not
   lock tables during that time* (with the exception of the "id"
   columns as described above). That means this should probably be run
   in `screen` or some other session that can be run for a long time.
   Notably, the migration will take *longer* than it would without
   these changes, but the website will still be responsive during that
   time.

These changes were tested on a relatively large statuses table (256k
entries), and the service remained responsive during the migration.
Migrations both forward and backward were tested.

* Rubocop fixes

* MigrationHelpers: Support ID columns in some cases

This doesn't work in cases where the ID column is referred to as a
foreign key by another table.

* MigrationHelpers: support foreign keys for ID cols

Note that this does not yet support foreign keys on non-primary-key
columns, but Mastodon also doesn't yet have any that we've needed to
migrate.

This means we can perform fully "concurrent" migrations to change ID
column types, and the IdsToBigints migration can happen with effectively
no downtime. (A few operations require a transaction, such as renaming
columns or deleting them, but these transactions should not block for
noticeable amounts of time.)

The algorithm for generating foreign key names has changed with this,
and therefore all of those changed in schema.rb.

* Provide status, allow for interruptions

The MigrationHelpers now allow restarting the rename of a column if it
was interrupted, by removing the old "new column" and re-starting the
process.

Along with this, they now provide status updates on the changes which
are happening, as well as indications about when the changes can be
safely interrupted (when there are at least 10 seconds estimated to be
left before copying data is complete).

The IdsToBigints migration now also sorts the columns it migrates by
size, starting with the largest tables. This should provide
administrators a worst-case scenario estimate for the length of
migrations: each successive change will get faster, giving admins a
chance to abort early on if they need to run the migration later. The
idea is that this does not force them to try to time interruptions
between smaller migrations.

* Fix column sorting in IdsToBigints

Not a significant change, but it impacts the order of columns in the
database and db/schema.rb.

* Actually pause before IdsToBigints
2017-10-02 21:28:59 +02:00
Akihiko Odaki bf575a1f5e Introduce recent to Follow (#3247)
Introduce recent to Follow, as Account and other models have.
This change also adds specs for the scope and the dependents.
2017-05-23 13:12:19 +02:00
yhirano 298796cc7b annotate models (#2697)
* add annotate to Gemfile

* rails g annotate:install

* configure annotate_models

* add schema info to models

* fix rubocop to add frozen_string_literal
2017-05-02 02:14:47 +02:00
Samy KACIMI 79ef756f64 fix rubocop issues 2017-04-05 00:47:17 +02:00
Samy KACIMI 81c76fe375 add more tests to models 2017-04-05 00:29:56 +02:00
Eugen Rochko de22c202f5 Add counter caches for a large performance increase on API requests 2017-03-30 15:06:59 +02:00
Eugen Rochko 0518492158 Stop trying to shoehorn all Salmon updates into the poor database-connected
StreamEntry model. Simply render Salmon slaps as they are needed
2017-02-12 01:19:14 +01:00
Eugen Rochko 50660d54e8 Fix semantics of follow requests another slaps 2017-02-11 17:13:11 +01:00
Eugen Rochko 8efa081f21 Remove Neo4J 2016-11-24 23:46:27 +01:00
Eugen Rochko e8c27767aa Remove orphaned notifications, add scopes param to app create API 2016-11-21 14:59:13 +01:00
Eugen Rochko fdc17bea58 Fix rubocop issues, introduce usage of frozen literal to improve performance 2016-11-15 16:56:29 +01:00
Eugen Rochko b13e7dda1f API pagination for all collections using Link header 2016-11-09 17:48:44 +01:00
Eugen Rochko 4d336cefac Add sync command for neo4j, fix configuration, add neo4j to docker-compose, fix seed 2016-10-15 12:37:43 +02:00
Eugen Rochko 20f581f796 Display follow suggestions 2016-10-15 12:06:30 +02:00
Eugen Rochko e21a3fe0cd Adding sync of follow relationships to Neo4J, accounts/suggestions API 2016-10-14 23:10:07 +02:00
Eugen Rochko 927333f4f8 Improve code style 2016-09-29 21:28:21 +02:00
Eugen Rochko 10ba09f546 Upgrade to Rails 5.0.0.1 2016-08-17 17:58:00 +02:00
Eugen Rochko a08e724476 Fix subscriptions:clear task, refactor feeds, refactor streamable activites
and atom feed generation to some extent, as well as the way mentions are
stored
2016-03-25 02:13:30 +01:00
Eugen Rochko 086d487145 Fix unfollows 2016-03-16 21:14:39 +01:00
Eugen Rochko fa7868675d Send Salmon interactions 2016-02-24 12:57:29 +01:00
Eugen Rochko fa33750105 Adding reblogs, favourites, improving atom generation 2016-02-23 19:17:37 +01:00
Eugen Rochko 1dad72bf13 Fixes and general progress 2016-02-22 18:10:30 +01:00
Eugen Rochko 709c6685a9 Made some progress 2016-02-22 16:00:20 +01:00