mastodon/app/models
David Yip 4a64181461 Allow keywords to match either substrings or whole words.
Word-boundary matching only works as intended in English and languages
that use similar word-breaking characters; it doesn't work so well in
(say) Japanese, Chinese, or Thai.  It's unacceptable to have a feature
that doesn't work as intended for some languages.  (Moreso especially
considering that it's likely that the largest contingent on the Mastodon
bit of the fediverse speaks Japanese.)

There are rules specified in Unicode TR29[1] for word-breaking across
all languages supported by Unicode, but the rules deliberately do not
cover all cases.  In fact, TR29 states

    For example, reliable detection of word boundaries in languages such
    as Thai, Lao, Chinese, or Japanese requires the use of dictionary
    lookup, analogous to English hyphenation.

So we aren't going to be able to make word detection work with regexes
within Mastodon (or glitchsoc).  However, for a first pass (even if it's
kind of punting) we can allow the user to choose whether they want word
or substring detection and warn about the limitations of this
implementation in, say, docs.

[1]: https://unicode.org/reports/tr29/
     https://web.archive.org/web/20171001005125/https://unicode.org/reports/tr29/
2017-10-21 14:54:36 -05:00
..
concerns Merge commit '4aea3f88a6d30f102a79c2da7fcfac96465ba1a8' into merging-upstream 2017-09-28 09:12:17 +02:00
form Default follows for new users (#4871) 2017-09-10 09:58:38 +02:00
web Make IdsToBigints (mostly!) non-blocking (#5088) 2017-10-02 21:28:59 +02:00
account_domain_block.rb Make IdsToBigints (mostly!) non-blocking (#5088) 2017-10-02 21:28:59 +02:00
account_filter.rb Improve admin UI for custom emojis, add copy/disable/enable (#5231) 2017-10-05 23:42:05 +02:00
account_moderation_note.rb foreign_key, non-nullable, dependent: destroy in account_moderation_notes (#5294) 2017-10-10 13:12:17 +02:00
account.rb Merge upstream 2.0ish #165 2017-10-11 10:43:10 -07:00
application_record.rb Fix rubocop issues, introduce usage of frozen literal to improve performance 2016-11-15 16:56:29 +01:00
block.rb Make IdsToBigints (mostly!) non-blocking (#5088) 2017-10-02 21:28:59 +02:00
context.rb Refactor JSON templates to be generated with ActiveModelSerializers instead of Rabl (#4090) 2017-07-07 04:02:06 +02:00
conversation_mute.rb Make IdsToBigints (mostly!) non-blocking (#5088) 2017-10-02 21:28:59 +02:00
conversation.rb Conditional validations no longer accept strings for if/unless (#3124) 2017-05-19 03:11:23 +02:00
custom_emoji_filter.rb Improve admin UI for custom emojis, add copy/disable/enable (#5231) 2017-10-05 23:42:05 +02:00
custom_emoji.rb Encode custom emojis as resolveable objects in ActivityPub (#5243) 2017-10-07 17:43:42 +02:00
domain_block.rb Make IdsToBigints (mostly!) non-blocking (#5088) 2017-10-02 21:28:59 +02:00
email_domain_block.rb Implement EmailBlackList (#5109) 2017-10-04 15:16:10 +02:00
export.rb Settings export refactor (#1646) 2017-04-13 13:02:02 +02:00
favourite.rb Make IdsToBigints (mostly!) non-blocking (#5088) 2017-10-02 21:28:59 +02:00
feed.rb Non-Serial ("Snowflake") IDs (#4801) 2017-10-04 09:56:37 +02:00
follow_request.rb Make IdsToBigints (mostly!) non-blocking (#5088) 2017-10-02 21:28:59 +02:00
follow.rb Make IdsToBigints (mostly!) non-blocking (#5088) 2017-10-02 21:28:59 +02:00
import.rb Make IdsToBigints (mostly!) non-blocking (#5088) 2017-10-02 21:28:59 +02:00
instance_filter.rb Add instance search feature (#4925) 2017-09-13 12:30:07 +02:00
instance.rb Add Instance class to list admin records (#3443) 2017-05-31 20:38:44 +02:00
keyword_mute.rb Allow keywords to match either substrings or whole words. 2017-10-21 14:54:36 -05:00
media_attachment.rb Merge upstream 2.0ish #165 2017-10-11 10:43:10 -07:00
mention.rb Make IdsToBigints (mostly!) non-blocking (#5088) 2017-10-02 21:28:59 +02:00
mute.rb Merge upstream 2.0ish #165 2017-10-11 10:43:10 -07:00
notification.rb Fix UserTrackingConcern firing on every request, optimize some queries (#5368) 2017-10-13 16:44:29 +02:00
preview_card.rb Increase max height of preview card image (#5092) 2017-09-25 17:10:49 +02:00
remote_follow.rb rescue HTTP::ConnectionError in RemoteFollowController#create (#4726) 2017-08-28 19:12:09 +02:00
remote_profile.rb Introduce OStatus::TagManager (#5008) 2017-09-19 18:08:08 +02:00
report_filter.rb Admin reports with accounts (#2092) 2017-04-18 19:36:18 +02:00
report.rb Make IdsToBigints (mostly!) non-blocking (#5088) 2017-10-02 21:28:59 +02:00
search.rb Refactor JSON templates to be generated with ActiveModelSerializers instead of Rabl (#4090) 2017-07-07 04:02:06 +02:00
session_activation.rb Refactor Web::PushSubscription, remove welcome message (#4524) 2017-09-01 13:35:23 +02:00
setting.rb Make IdsToBigints (mostly!) non-blocking (#5088) 2017-10-02 21:28:59 +02:00
site_upload.rb Uploads for admin site settings (#4913) 2017-09-14 00:04:30 +02:00
status_pin.rb Allow multiple pinned statuses to be shown and make them be ordered b… (#4690) 2017-08-25 18:50:52 +02:00
status.rb Merge remote-tracking branch 'origin/master' into gs-master 2017-10-16 01:29:02 -05:00
stream_entry.rb Merge upstream 2.0ish #165 2017-10-11 10:43:10 -07:00
subscription.rb Make IdsToBigints (mostly!) non-blocking (#5088) 2017-10-02 21:28:59 +02:00
tag.rb add validation to tag name (#4194) 2017-07-14 11:02:49 +02:00
user.rb Add option to reduce motion (#5393) 2017-10-16 09:36:15 +02:00
web.rb Add extended about page stub 2017-01-13 03:24:41 +01:00