Commit Graph

4783 Commits

Author SHA1 Message Date
David Yip 4a64181461 Allow keywords to match either substrings or whole words.
Word-boundary matching only works as intended in English and languages
that use similar word-breaking characters; it doesn't work so well in
(say) Japanese, Chinese, or Thai.  It's unacceptable to have a feature
that doesn't work as intended for some languages.  (Moreso especially
considering that it's likely that the largest contingent on the Mastodon
bit of the fediverse speaks Japanese.)

There are rules specified in Unicode TR29[1] for word-breaking across
all languages supported by Unicode, but the rules deliberately do not
cover all cases.  In fact, TR29 states

    For example, reliable detection of word boundaries in languages such
    as Thai, Lao, Chinese, or Japanese requires the use of dictionary
    lookup, analogous to English hyphenation.

So we aren't going to be able to make word detection work with regexes
within Mastodon (or glitchsoc).  However, for a first pass (even if it's
kind of punting) we can allow the user to choose whether they want word
or substring detection and warn about the limitations of this
implementation in, say, docs.

[1]: https://unicode.org/reports/tr29/
     https://web.archive.org/web/20171001005125/https://unicode.org/reports/tr29/
2017-10-21 14:54:36 -05:00
David Yip 2e03a10059 Spike out index and new views for keyword mutes controller. 2017-10-21 14:54:36 -05:00
David Yip 4fa2f7e82d Set up /settings/keyword_mutes. #164.
This should eventually be accessible via the API and the web frontend,
but I find it easier to set up an editing interface using Rails
templates and the like.  We can always take it out if it turns out we
don't need it.
2017-10-21 14:54:36 -05:00
David Yip b4b657eb1d Invalidate cached matcher objects on KeywordMute commit. #164. 2017-10-21 14:54:36 -05:00
David Yip 693c66dfde Use more idiomatic string concatentation. #164.
The intent of the previous concatenation was to minimize object
allocations, which can end up being a slow killer.  However, it turns
out that under MRI 2.4.x, the shove-strings-in-an-array-and-join method
is not only arguably more common but (in this particular case) actually
allocates *fewer* objects than the string concatenation.

Or, at least, that's what I gather by running this:

    words = %w(palmettoes nudged hibernation bullish stockade's tightened Hades
    Dixie's formalize superego's commissaries Zappa's viceroy's apothecaries
    tablespoonful's barons Chennai tollgate ticked expands)

    a = Account.first

    KeywordMute.transaction do
      words.each { |w| KeywordMute.create!(keyword: w, account: a) }

      GC.start

      s1 = GC.stat

      re = String.new.tap do |str|
        scoped = KeywordMute.where(account: a)
        keywords = scoped.select(:id, :keyword)
        count = scoped.count

        keywords.find_each.with_index do |kw, index|
          str << Regexp.escape(kw.keyword.strip)
          str << '|' if index < count - 1
        end
      end

      s2 = GC.stat

      puts s1.inspect, s2.inspect

      raise ActiveRecord::Rollback
    end

vs this:

    words = %w( palmettoes nudged hibernation bullish stockade's tightened Hades Dixie's
    formalize superego's commissaries Zappa's viceroy's apothecaries tablespoonful's
    barons Chennai tollgate ticked expands
    )

    a = Account.first

    KeywordMute.transaction do
      words.each { |w| KeywordMute.create!(keyword: w, account: a) }

      GC.start

      s1 = GC.stat

      re = [].tap do |arr|
        KeywordMute.where(account: a).select(:keyword, :id).find_each do |m|
          arr << Regexp.escape(m.keyword.strip)
        end
      end.join('|')

      s2 = GC.stat

      puts s1.inspect, s2.inspect

      raise ActiveRecord::Rollback
    end

Using rails r, here is a comparison of the total_allocated_objects and
malloc_increase_bytes GC stat data:

                 total_allocated_objects        malloc_increase_bytes
string concat    3200241 -> 3201428 (+1187)     1176 -> 45216 (44040)
array join       3200380 -> 3201299 (+919)      1176 -> 36448 (35272)
2017-10-21 14:54:36 -05:00
David Yip a4851100fd Make use of the regex attr_reader. #164.
It would also have been valid to get rid of the attr_reader, but I like
being able to reach inside KeywordMute::Matcher without resorting to
instance_variable_get tomfoolery.
2017-10-21 14:54:36 -05:00
David Yip 9f609bc94e Fix case-insensitive match scenario; test some word ornamentation. #164. 2017-10-21 14:54:36 -05:00
David Yip 603cf02b70 Rework KeywordMute interface to use a matcher object; spec out matcher. #164.
A matcher object that builds a match from KeywordMute data and runs it
over text is, in my view, one of the easier ways to write examples for
this sort of thing.
2017-10-21 14:54:36 -05:00
David Yip 4745d6eeca Spec out KeywordMute interface. #164. 2017-10-21 14:54:21 -05:00
David Yip 9093e2de7a Add KeywordMute model.
Gist of the proposed keyword mute implementation:

Keyword mutes are represented server-side as one keyword per record.
For each account, there exists a keyword regex that is generated as one
big alternation of all keywords.  This regex is cached (in Redis, I
guess) so we can quickly get it when filtering in FeedManager.
2017-10-21 14:53:41 -05:00
Ondřej Hruška d589dd7cd0 Compose buttons bar redesign + generalize dropdown (#194)
* Generalize compose dropdown for re-use

* wip stuffs

* new tootbox look and removed old doodle button files

* use the house icon for ...
2017-10-21 20:24:53 +02:00
beatrix a7be86e875 hide mentions of muted accounts (in home col) (#190)
* hide mentions of muted accounts (in home col)

also cleans up some old crap

* add test
2017-10-20 10:49:54 -04:00
beatrix b15dd05514 Merge pull request #191 from glitch-soc/garglamel-yaml
ƔAML update
2017-10-19 19:29:52 -04:00
kibigo! 21bafc6555 Updates to bio metadata script 2017-10-19 16:11:53 -07:00
beatrix f5e2469485 Merge pull request #189 from glitch-soc/scrollable-compose-area
Make the compose area optionally scrollable
2017-10-19 12:28:47 -04:00
David Yip 9423553e5c Make the compose area optionally scrollable.
On desktop, the compose text box grows to accommodate the content.  On
mobile, the text box does not grow to accommodate text context, but does
grow to accommodate images.  It is possible in both cases to overflow
the available area, which makes accessing other UI elements (e.g.
visibility setttings) difficult.

This commit makes the compose area optionally scrollable, which allows
those UI elements to remain available even if they go off-screen.
2017-10-19 10:59:50 -05:00
David Yip 90770f6d59 Merge pull request #185 from glitch-soc/fix-null-status
workaround for null status
2017-10-18 17:01:31 -05:00
beatrix c756651278 Merge pull request #188 from glitch-soc/merge-upstream-2-0-0
Merge with upstream v2.0.0
2017-10-18 17:57:42 -04:00
Ondřej Hruška eb907a5bab
formatting fix for eslint 2017-10-18 23:44:06 +02:00
David Yip 39c9cdf7fe Remove unused filesAttached property. 2017-10-18 13:20:45 -05:00
David Yip 86cf4468af Update stylesheet imports in glitch components.
Commit 6e54719474 moved the Mastodon
variables and mixins deeper in the directory hierarchy; this commit
brings the glitch components in line with that change.
2017-10-18 11:52:34 -05:00
David Yip 42e8c8eb0e Merge tag 'v2.0.0' into gs-master 2017-10-18 11:52:04 -05:00
Sho Kusano 09d81defcd Suppress type error(not a function) on calling fastSeek (#5452) 2017-10-18 17:13:51 +02:00
Eugen Rochko 3810d98cd8 Bump version to 2.0.0 🐘 2017-10-18 13:53:56 +02:00
Renato "Lond" Cerqueira 26b2a6a71e Fix pt-BR translation strings related to advanced search. (#5449) 2017-10-18 13:53:17 +02:00
Technowix edf9a5e4fc Revert #5438 for FR (#5450)
As said here https://github.com/tootsuite/mastodon/pull/5438 the point of shortening the timestamp is legit, and after some time of adaptation no mistakes can be mades.
2017-10-18 13:51:30 +02:00
Håkan Eriksson c710069c12 Some typos and supplementation in sentence structures (#5441)
* Swedish file added

* Swedish file added

* Swedish file updated

* Swedish languagefile added

* Add Swedish translation

* Add Swedish translation

* Started the Swedish translation

* Added Swedish lang settings

* Updating Swedish language

* Updating Swedish language

* Updating Swedish language

* Updating Swedish language

* Updating Swedish language

* Updating Swedish language

* Swedish language completed and added

* Swedish language Simple_form added

* Swedish language Divise added

* Swedish language doorkeeper added

* Swedish language - now all file complete

* Swedish - Typos and supplementation in sentence structure

* Update simple_form.sv.yml

* Update sv.yml

* Update sv.yml

Rearranged the alphabetical order.
2017-10-18 13:50:52 +02:00
Yamagishi Kazutoshi 990d6dd565 Run `i18n-tasks checked-normalized` in Travis CI (#5443) 2017-10-18 11:57:02 +02:00
Yamagishi Kazutoshi 402da46ff6 Enable coverage for Jest (#5442) 2017-10-18 11:39:36 +02:00
JohnD28 e7099d8d9e fr.json typo (realtive time) (#5447)
Typo correction : https://fr.wikipedia.org/wiki/Heure#Typographie
2017-10-18 16:47:14 +09:00
Eugen Rochko 637ea3bb5b Bump version to 2.0.0rc4 2017-10-17 23:16:35 +02:00
Jeroen 363d0d3a44 Dutch: A few strings I found last minute (#5440)
* Dutch: A few strings I found last minute

* Update simple_form.nl.yml
2017-10-17 23:15:40 +02:00
YaQ 6e54719474 Make theme directory and Rename application.scss (#5336)
* Rename application.scss

* Move to 'default' directory

* Follow review

Move directory, and Rename "mastodon".

* Revert rename

* undo removal of newline
2017-10-18 05:20:07 +09:00
Eugen Rochko f3003417c5 When unreblog arrives over streaming API, just delete in UI (#5439) 2017-10-17 22:17:02 +02:00
JeanGauthier 33ea042dec i18n better wording (OC/CA/FR) + string added in FR (#5438)
* Update oc.json

* Update fr.json

* Update ca.json

* Update fr.json

* Update simple_form.fr.yml

* Update ca.json

* Update fr.json

* Update oc.json

* Update oc.json

* Update fr.json
2017-10-17 21:49:39 +02:00
Nolan Lawson 8b22a63ab0 Fix scrolling of dropdown when offscreen (#5434) 2017-10-17 20:38:16 +02:00
Eugen Rochko 05686cc99d Fix unwanted content warning gap in CSS (#5436)
Fix #5356
2017-10-17 20:07:44 +02:00
Eugen Rochko 484208ce12 When status is fetched instead of delivered, do not stream it (#5437) 2017-10-17 20:05:21 +02:00
unarist 3bc8924940 Don't capture scheme-less URLs in the status (#5435)
Specifically, this fixes status length calculation to be same as JS side.

BTW, since this pattern used in not only preview card fetching, we
should extract it (with twitter-regex?) and write tests I think.
2017-10-17 18:32:25 +02:00
JeanGauthier a02de9e012 i18n update (#5427) in OC/CA/FR/ES (#5431)
* Update oc.json

* Update ca.json

* Update fr.json

* Update es.json
2017-10-17 13:03:28 +02:00
unarist 2d395324e1 Fix React warning about tabIndex on status with CW (#5432) 2017-10-17 13:03:12 +02:00
Jakob Kramer e6c9756fa9 Update German translation (#5425) 2017-10-17 12:02:47 +02:00
Yamagishi Kazutoshi 5050719fac Add Japanese translations (#5427)
* yarn manage:translations

* Add Japanese translation for #5410

* Add Japanese translation for #5393
2017-10-17 11:47:05 +02:00
Yamagishi Kazutoshi 989553c69a Normalize locale files (#5429) 2017-10-17 11:45:59 +02:00
masarakki 0e0c6b1b4b use-DB_NAME-in-development (#5430) 2017-10-17 11:45:37 +02:00
aschmitz 554c2fd8af Clean up reblog tracking keys, related improvements (#5428)
* Clean up reblog-tracking sets from FeedManager

Builds on #5419, with a few minor optimizations and cleanup of sets
after they are no longer needed.

* Update tests, fix multiply-reblogged case

Previously, we would have lost the fact that a given status was
reblogged if the displayed reblog of it was removed, now we don't.

Also added tests to make sure FeedManager#trim cleans up our reblog
tracking keys, fixed up FeedCleanupScheduler to use the right loop,
and fixed the test for it.
2017-10-17 11:45:06 +02:00
Håkan Eriksson a2b600428c Swedish translation (#5406)
* Swedish file added

* Swedish file added

* Swedish file updated

* Swedish languagefile added

* Add Swedish translation

* Add Swedish translation

* Started the Swedish translation

* Added Swedish lang settings

* Updating Swedish language

* Updating Swedish language

* Updating Swedish language

* Updating Swedish language

* Updating Swedish language

* Updating Swedish language

* Swedish language completed and added

* Swedish language Simple_form added

* Swedish language Divise added

* Swedish language doorkeeper added

* Swedish language - now all file complete
2017-10-17 13:41:36 +09:00
Jeroen df1a9c5ab5 Small update Dutch (nl) strings (#5424)
* Update Dutch strings

* Update Dutch
2017-10-17 01:00:23 +02:00
Ondřej Hruška 4421f6598f
Fix dual toot button wrong behavior after compose re-design 2017-10-16 23:32:13 +02:00
Ondřej Hruška 64a5e9a1de
workaround for null status 2017-10-16 23:09:39 +02:00