Gitter feature parity overview issue

[MadLittleMods]

In the long-run we want Element to take-over for Gitter because we don't want to support both platforms. In order to have Element be a suitable successor, we want to make sure all of the feature functionality people are used to on Gitter is available in Element. If you want to read more context behind this, the Matrix blog post has great detail. All of the Gitter content will be available on the gitter.im Matrix homeserver to live on.

Feel free to edit this list with more features, details, and links(MSC's, etc) as you think of them. We can split this out individual issues once this list has a few more iterations.

Gitter features needed in Element

• Communities
  • MSC: This is currently being implemented in Matrix as spaces MSC1772: Matrix spaces matrix-org/matrix-spec-proposals#1772
• Backfill existing message history
  • MSC2716
  • Gitter has a huge archive of message history and we want to migrate all of that over to the Matrix rooms
  • Need to increase the volume size on the gitter.im homeserver to accommodate all of the messages, https://github.com/matrix-org/matrix-hosted/issues/3865, https://github.com/matrix-org/matrix-hosted/issues/6793#issuecomment-1357924751
  • Tune Synapse to handle the import load, https://github.com/matrix-org/matrix-hosted/issues/6793
  • The actual import process:
    • Import script: https://gitlab.com/gitterHQ/webapp/-/merge_requests/2313
    • https://gitlab.com/gitterHQ/webapp/-/issues/2609
    • https://gitlab.com/gitterHQ/webapp/-/issues/2857
    • https://gitlab.com/gitterHQ/webapp/-/issues/2858
    • Do we worry about notifications and emails?
      • Emails for Twitter are currently not updated with the Twitter SSO config in Synapse (https://github.com/matrix-org/matrix-hosted/issues/6901)
      • Not worried about it. Active users will be able to sign-in to Element and update their email as needed
• Threading (threaded conversations)
  • MSC3440
  • MSC2836
  • Pretty much already supported with the way replies work in Element. Also potential to just pull them all out into a sidebar just like Gitter.
• Support Math/LaTeX message syntax
  • MSC2191
  • There is a Render LaTeX maths in messages lab feature in Element
• GitLab/GitHub based permissions for a room
  • MSC: SSO logins are being looked at in MSC2858: Multiple SSO Identity Providers matrix-org/matrix-spec-proposals#2858 (social login)
  • https://ems.element.io/guides/openid-connect#gitlab
  • This doesn't seem strictly necessary. We can make rooms private and the permissions can flow from spaces to inherit.
  • The Gitter GH_REPO_PUSH permission for admins on Gitter was always a bit flawed and too permissive to all of the contributors. People often switched this over to manual to give only a select few control of the Gitter room.
• GitLab/GitHub activity as a first-class citizen in a room’s side-panel
  • All of the widget stuff happening in the Element/Matrix side these-days, seems perfect
  • https://github.com/Half-Shot/matrix-hookshot
  • Activity sidepanel widget (gitter-like) matrix-org/matrix-hookshot#631
• GitHub/GitLab issue decoration (popover of title/description when you hover link, open/closed/merged -> green/red/purple status)
  • Not much of a value add 🤷‍♀️. We could improve the link previewing if it's not great for GitLab/GitHub
• Performant load times (seconds to load into Element)
  • Bundle size
  • Lazy load
  • There have been some huge improvements recently although this could still be better.
• Public static archive (indexed by search engines)
  • Update: This is now covered by https://archive.matrix.org/
    • https://github.com/matrix-org/matrix-public-archive is now taking up the mantle here 🏃
  • Similar to https://view.matrix.org/
  • URL for an arbitrary day of history and navigation for next and previous days element-web#7677
  • Related to MSC3030
  • Use Hydrogen to server-side render a page with all of the content for the date given
• Anonymous users(NLI) can also view a room
  • This is already in Element, just visit https://app.element.io/#/room/#matrix:matrix.org
• Sidecar for embedding a chat room on your own website
  • Add embeddable view of rooms for people to put on their own websites matrix-org/matrix-viewer#157

Extras

• Bridge information spec for indicating room is from Gitter in the Element UI
  • MSC: MSC2346: Bridge information state event matrix-org/matrix-spec-proposals#2346
• User badges on the Matrix-side to show that user is from the Gitter network
  • We can use the existing iteration of community flairs
  • Related MSC: https://github.com/matrix-org/matrix-doc/pull/1772/files#r520808696
• Portable identities so MXID's can match the Gitter username and we can handle renames seamlessly, MSC2787: Portable Identities matrix-org/matrix-spec-proposals#2787
  • Currently, we have to have MXID's as @username-id:gitter.im and we just key off of the Mongo ID
  • This could also allow people to control their @*:gitter.im MXID from their real Matrix account and appear completely normal on Gitter (as discussed at https://gitter.im/gitter/gitter?at=5ff8f0befe007479e4fda491)
• MSC2162: Signaling Errors at Bridges, MSC2162: Signaling Errors at Bridges matrix-org/matrix-spec-proposals#2162
• Room peeking (see room messages without joining) (peeking over federation)
  • MSC: MSC2753: Peeking via sync (take 2) matrix-org/matrix-spec-proposals#2753

Sunset

After Gitter shuts down, we want the experience of accessing your same Gitter content on Matrix just as seamless.

• Let people free-for-all on the gitter.im homeserver (another free matrix.org like server)
  • People login with social like before and an application service (AS) can invite them to the same room membership, https://github.com/vector-im/sre-internal/issues/1754, https://github.com/matrix-org/matrix-hosted/issues/6901
  • Gitter branded Element instance, https://github.com/matrix-org/matrix-hosted/issues/6824
  • Setup redirects to rewrite the old Gitter URL to Matrix room alias, https://github.com/vector-im/sre-internal/issues/1755
  • As discussed in an internal sync (2022-05-19)
  • Also discussed in this internal sync (2022-12-01)

Previous after Gitter sunset idea: push people to their own homeserver

• Sign in with GitLab, GitHub, Twitter -> automatically bootstrap their matrix.org account and be signed in
  • This way, we don’t have to pay for beefing up the gitter.im homeserver with people trying to join big rooms
  • We want the experience to be easy for user with no knowledge of Matrix. They should just need to sign in with GitLab/GitHub like they did with Gitter and we will handle the Matrix user creation and plop them in. This will also add admin permissions for people in the rooms they were admins in.
  • People may sign in and get the automatic Matrix.org account created but want to access their rooms from another Matrix account. We should also have a migration utility to be able /migrate_room_membership_to @xxx and have it invite their other user to the same rooms.
    • EMS has a generic MXID migration tool we can point people to, https://ems.element.io/tools/matrix-migration
  • Instead of pushing everyone to matrix.org, maybe suggest EMS for your own homeserver or maybe P2P if that's mature in time. The one problem with this flow is it would probably be super cumbersome and clumsy for someone not familiar with Matrix, e.g. "Why do I have to create a server if I just want to sign into Gitter?"

Master migration rollout plan

1. Import all messages from Gitter -> Matrix, https://gitlab.com/gitterHQ/webapp/-/issues/2609
2. Sync membership in all live Matrix rooms, https://gitlab.com/gitterHQ/webapp/-/issues/2857
3. Sync membership in all historical Matrix rooms, https://gitlab.com/gitterHQ/webapp/-/issues/2857
4. Sync admin permissions to all live and historical Matrix rooms
5. Pre-announcement cutover blog post, https://docs.google.com/document/d/1WNuQ3s8dNn7uxdkG515Hnl07RhnCrXC7jmUP1r6czJA/edit#
   -> https://blog.gitter.im/2023/01/16/gitter-is-going-fully-native-matrix-in-feb-2023/
6. Link pre-announcement from announcement section of the left-menu
7. Ensure live room points back to historical room via MSC3946,
   • https://gitlab.com/gitterHQ/webapp/-/issues/2858
   • Epic: Support Gitter sunsetting element-web#24262
8. Inject social provider information into Synapse, https://github.com/vector-im/sre-internal/issues/1754
9. Setup app.gitter.im with the Gitter branded Element instance, https://github.com/matrix-org/matrix-hosted/issues/6824
10. Setup /.well-known/matrix/client so people can enter gitter.im on the Element sign-in page
    • Add /.well-known/matrix/client for Matrix clients in the webapp, https://gitlab.com/gitterHQ/webapp/-/merge_requests/2336
    • https://github.com/vector-im/sre-internal/issues/1754
    • https://github.com/vector-im/sre-internal/issues/1781
11. Update Matrix appservice registration to not have exclusive control over the users (make sure updated on EMS), https://github.com/matrix-org/matrix-hosted/issues/6979
12. Remove custom Synapse patches used to optimize during the import process, https://github.com/matrix-org/matrix-hosted/issues/7144
13. Deploy latest Element which includes some updates for Gitter, https://github.com/matrix-org/matrix-hosted/issues/7148
14. Allow anonymous preview of rooms on app.gitter.im, https://github.com/matrix-org/matrix-hosted/issues/7150
15. Enable social login on app.gitter.im
16. Blog post explaining the Gitter -> Element/Matrix transition, https://docs.google.com/document/d/1s05BHAz-EFKktfBZ3BD2j5ojR2xIwDR13t_6XiNdI_M/edit#
    1. FAQ document we can link wherever
    2. -> https://blog.gitter.im/2023/02/13/gitter-has-fully-migrated-to-matrix/

--- Cutover on 2023-02-13 ---

1. Update Gitter homepage to direct people to app.gitter.im so all new people go in via Matrix
   • New design: https://github.com/vector-im/design/issues/35
   • Can we host this not on current AWS infra as some static HTML (static site generator)? https://github.com/vector-im/sre-internal/issues/1781
   • -> Updating the webapp in the interim: https://gitlab.com/gitterHQ/webapp/-/merge_requests/2401
2. Disallow sign-in to Gitter itself. Update sign-in page to point to app.gitter.im to better direct Gitter desktop users to the right place
   • -> https://gitlab.com/gitterHQ/webapp/-/merge_requests/2404
3. Also need to stop people generating tokens from their already created OAuth apps. Probably cut-off token exchange endpoint.
   • -> https://gitlab.com/gitterHQ/webapp/-/merge_requests/2404
4. Disallow sign-in on https://developer.gitter.im/ to stop people from generating new tokens
   • -> https://gitlab.com/gitterHQ/developer-gitter-im/-/merge_requests/37
5. Again -> Inject social provider information into Synapse, https://github.com/vector-im/sre-internal/issues/1754
   • Resume dump from gitterUserId=63bc7a736da0373984af769a (small amount of overlap from our last dump)
   • Resume dump from gitterUserId=63e9a3486da0373984be49f7
6. Invalidate all Gitter access tokens so people get a 401 and are redirected back to the home page. People will then stumble back in via app.gitter.im
   • This will result in a confusing hard cut-off for the Gitter iOS and Android apps which have been deprecated since 2020-11-20
   • When we invalidate all tokens, this will also break people's scripts/bots. Might just have to rip off the band-aid on this one.
7. Setup redirects from old Gitter URL scheme to Element/Matrix location, https://github.com/vector-im/sre-internal/issues/1755, https://gitlab.com/gitterHQ/gitter.im/-/issues/9
   • -> https://gitlab.com/gitterHQ/webapp/-/merge_requests/2406
8. Stop Gitter unread emails from going out, https://gitlab.com/gitterHQ/gitter-infrastructure/-/issues/261
9. Shutdown the IRC bridge, https://gitlab.com/gitterHQ/gitter-infrastructure/-/issues/262
10. Remove Android and iOS apps from the app stores, https://github.com/vector-im/sre-internal/issues/1854
11. Stop all public traffic to AWS Gitter infra
    • Be mindful that people still need to access the Gitter homepage in case we're still serving that from the Gitter infra

--- Cleanup ---

1. Archive Gitter Mongo data, https://github.com/vector-im/sre-internal/issues/1929
2. Shutdown AWS Gitter infra
3. Cancel and stop paying for associated services, https://github.com/vector-im/sre-internal/issues/1927
4. Archive projects, https://gitlab.com/gitterHQ

[NickHu]

I think it's a bit misleading to mark the Maths feature as done. For web it works, but I need help finishing element-hq/element-android#2133, and as far as I'm aware nobody has started implementing it on iOS. My PR has stalled for over a year now, and I'm quite sure that it doesn't need much developer time to merge.

[marclaporte]

It's happening!

"So, after over 2 years, we’re pressing the button at last: we’re migrating all the existing content in the original centralised Gitter platform over into Gitter’s Matrix server, and we’re planning to replace the old Gitter app with a Gitter-customised version of Element during the week of Feb. 6th 2023."

https://blog.gitter.im/2023/01/16/gitter-is-going-fully-native-matrix-in-feb-2023/

[csarven]

Setup redirects to rewrite the old Gitter URL to Matrix room alias, https://github.com/vector-im/sre-internal/issues/1755

That URL is 404 for the public or at least I'm not authorized to read. Can the comment be updated to refer to the issue/PR addressing redirect or some information about its status?

[MadLittleMods]

@csarven The links are up-to-date but those are internal repositories that you won't have access to.

[melvincarvalho]

The sunset period is FAR too short. Please consider extending it.

[timbl]

So now I have the Element app on Mac, ... I see one room is in bridged from Gitter, but I don't see any other Gitter rooms. How to I open an arbitrary Gitter room in Element? How from Element can I see what Gitter rooms exist?