Discourse

Ongoing evaluation of Discourse for Forums / Mailing List needs.

https://www.discourse.org

Old proposal thread: https://markmail.org/thread/rnmyc5upbxdqoug5

Trial

http://discuss.kde.org.uk Give it a shot. Testbed server from June 2019. Set up by Jonathan.

Pile of Requirements/Desires

Easy and quick to use spamer nuke feature
Mark threads as solved
Forum banner / Newsbar banner
Guided posting wizard
Profile Icons to show OS/Distribution
Brainstorm (post voting)
Identity Integration (auth)
Scalable search for amount of posts we have
Tagcloud
Gallery (for krita; unclear of what is exactly required here)
Retain all data from current forum

Stack Eval

Look into the actual technology stack they use (seems to be Rails based in this case) to make sure there aren't any potential snags there

Ben took a quick look: My main one [concern] here is the lack of any options for installation other than Docker which makes no sense for a Rails application. Looking into their Docker image installation script I see that they build both Nginx and Imagemagick themselves (and stepping outside of package repositories is generally a bad idea). Imagemagick is of grave concern as this project has had numerous security advisories in the past and I see the version they're using isn't the latest one. I have further concerns for Nginx as they include a third party compression module, Brotli, whose codebase hasn't been touched in 2 years (plus it's a compression method, so you have the risk of CRIME/BREACH attacks).

Auth Options

Evaluate what support it has for authentication options (Identity requires LDAP at the moment, but will move to OAuth2 at some point using a custom API)

For identity this might be useful https://github.com/jonmbake/discourse-ldap-auth
There is also the option of writing an SSO provider to abstract discourse<>provider>[oauth;identity] https://meta.discourse.org/t/official-single-sign-on-for-discourse-sso/13045
Simple oauth https://github.com/discourse/discourse-oauth2-basic
Custom oauth plugin https://meta.discourse.org/t/login-to-discourse-with-custom-oauth2-provider/14717/3
Fully custom auth plugin https://meta.discourse.org/t/vk-com-login-vkontakte/12987

Data Import

Determine what's needed to import existing data we have

We have phpbb 3.0 which is supported to migrate from. There may be problems with custom mods adding custom stuff to the database (notably the OS/distro icon would not be migrated obviously). It may be good to actually have a schema to look at. Also, hard to tell how well this will work in practice without giving it a try with an actual db dump from the production phpbb forum.

Guide

Structure

Ascertain how best to structure things to make it easy for end-users to work with.

One would presume the structure could be very/entirely similar to what we have currently. Discourse offers a fairly similar view, scalable to many different subforums (e.g. https://discourse.ubuntu.com/). It may be wise to also revisit the overall structure and possibly merge some forums though.

Anti-Spam

Investigate what anti-spam options are available and how maintainable any customisations we need to support KDE specific workflows will be

Discourse actually has built-in spam protection aid in the form of trust levels which prevent new users from doing spammy things in general https://blog.discourse.org/2018/06/understanding-discourse-trust-levels/
On top of trust levels there's also a flagging system which auto hides posts (possibly not the most up-to-date: https://meta.discourse.org/t/so-what-exactly-happens-when-you-flag/275/3)
Overview of the above two points https://meta.discourse.org/t/what-about-the-spam-problem/2724/8
There is also built-in monitoring which allows admins to view a list of "suspicious" users by applying a bunch of metrics to determine if a (new) user may be a spammer (/admin/users/list/suspect)
Discourse also has built-in screening capabilities where apparently all sorts of stuff can be used to block or mark posts for review. This at least includes originator IP (ranges), email addresses patterns and URL patterns.
easy nuking of users and their posts; looks like this: https://meta.discourse.org/t/new-user-deleted-for-spam-posts/53647/2
Akismet https://www.discourse.org/plugins/akismet.html probably no bueno because akismet is a paid service and sends data off a third party service which may be problematic WRT our privacy policy

TODO

Figure out how our current forums' spam protection works. [sitter: the spam todo we can probably tick off. from what I've seen on the demo instance the protection capabilities should be about en-par with what we have on the phpbb side, in fact probably better because of the entire trust system limiting what new accounts can do]
Talk to upstream about their awareness of docker security responsibility and the specific issues Ben highlighted on the mailing list.
Get some test setup and figure out how to structure stuff (as in: how to mimic subforums etc)
Look into how the various subforums utilize phpbb and if that still works with discourse
Figure out what to do about the custom mods we have on phpbb (generally we'd want to get rid of as much as possible to increase maintainability etc?) ( Krita customisations to preview first image post https://forum.kde.org/viewforum.php?f=275 )
Do we also migrate mailing lists to discourse? separate developers and end user categories
Do we migrate mailing lists to the same discourse instance or separate user forums from developer forums by using separate instances?
For mailing lists it's common to already have a list of e-mails of people to add in. Can we add in e-mails into a Category in Discourse or does each user need to faff around adding themselves? invite method?
Account integration with identity.kde.org? https://www.discourse.org/plugins/oauth.html
Can it support multiple domains for incoming e-mails?
Can we import threads from forum.kde.org phpBB ? https://meta.discourse.org/t/importing-from-phpbb3/30810
For a forum with an incoming e-mail address how does a user find out what that address is?
What are the tools to remove identifying info for users who request it?
Command line tool to approve or not posts?
IRC/Matrix announce bot (Persuvant for commit notification etc works fine)
Guided Posting - equivalent to custom phpbb tool to guide users to correct forum to post in
Can you have sub-forums with labels how Krita and others to on forums?
Find projects to test moving to it e.g. kmymoney or gcompris or kde neon
Possiblity to not deploy on Docker?