Skip to content

MSC4278: Media preview controls #4278

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 16 commits into
base: main
Choose a base branch
from
Open
Changes from 11 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
157 changes: 157 additions & 0 deletions proposals/4278-media-preview-switch.md
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Implementation requirements:

  • Client

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

element-hq/element-web#29582 should now be competent. The remaining piece was spaces, but we iterated and removed spaces from the MSC instead.

Original file line number Diff line number Diff line change
@@ -0,0 +1,157 @@
# MSC4278: Media preview controls

Matrix caters to a wide variety of networks configurations. Some networks are closed and the users have a high
degree of safety, and other networks are open to the public internet and are inheriently less safe. This proposal
aims to give the user more universal control over the content they see on Matrix, starting with media previews.

Most graphical Matrix clients display media in the timeline of a room with a preview. Often these are delivered
without any consent prompt, at least by default. This presents a risk to users who may be joining the network
and do not have the same safety barriers that experienced users may have configured.

With that in mind, this MSC provides a simple switch for users and homeservers to configure a global media preview
preference.

For the purposes of this MSC, the term "undesirable content" refers to images that the user does not want to see,
be it illegal, abusive or otherwise wanted.


## Proposal

This proposal introduces a new account data key, `m.media_preview_config`.

This key contains the following content.

```json
{
"media_previews": "off|on|private",
"invite_avatars": "off|on"
}
```

`media_previews` refers to media in the room timeline, that may be thumbnailed.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this include user avatars, or media in reactions from MSC4027? The text should probably spell that out explicitly. If they are not included, we should consider separate fields to control them. User avatars, in particular, are a common abuse vector in my experience. Most current clients that have a "hide media" option do not hide user avatars.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1 to spelling it out.

It's a good question. This originally focused on timeline media, either m.image/m.video, m.sticker, or inline media but reactions and avatars are other possible abuse vectors.

My feeling is that for reactions, it might just be suitable to show the alt text in public rooms. For user avatars, I think losing all avatars in a public room might be a bit extreme. One way we could go about this is allowing users to click-to-consent to individual avatars in public rooms like we have for Element's previews, and then that is tracked. We'd need some way to cascade rules from spaces so that people in my trusted circle are automatically permitted.

And of course the problem with all this is the client needs to clearly tell you why an avatar is/isn't displaying to prevent it looking like a bug.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, that all makes sense, and I think points towards having a separate field for hiding avatars. Personally, there are some rooms that get frequent spam with abusive avatars where I would 100% turn on "hide all avatars" if it was an option in the client. The ability to selectively show avatars from trusted users (or even manually enabling avatars for everybody in the room periodically when there isn't spam happening) would be preferable. I wouldn't want to bog down this MSC with the complexity of per-user switches though, and am happy with incremental improvement.


`invite_avatars` refers to any *room* avatar rendered in the client, where the client has a `membership` of `invite`.

The fields `media_previews` and `invite_avatars` may be one of three values.

#### `off`

The client MUST not show any previews for any media, in the room.

Users may individually consent to seeing media, for example by clicking on a prompt to show a preview.

If consent is given, the client SHOULD then track that consent and show the media again in the future.

#### `on`
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Other platforms (discord for example) offers more than just on/off. It offers on/blur/block.
I think that it is a good addition. Clients that do not support blur could just block in that case.

The blured image gives a rough idea of the image without exposing the user to the real image.

Without that how is a user expected to react to a off image? Never click on it or take the risk and click on it?

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The main trouble with blur is that the picture is fetched and ends up in both the client and the server cache. It's quite bad if it's illegal content.

We may still want the blur option for private federations however ? I wouldn't recommend that as a default for the public federation because of ^.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

blurhashes do not, as they're part of the event's metadata #2448 (unless it's user avatars, which is a different kettle o fish)

But I think it's probably subjective whether there is enough information in a blurred image to tell you whether something is NSFW. Also, clients can lie about the hash so if anything it's likely to be abused as much as avatars.

I'd probably say blurs are really just useful as loading indicators but otherwise should be for trusted media.


Media MAY be shown in any room without a prompt.

Users may individually hide media, and this preference MUST be respected over any defaults defined in `m.media_preview_config`.

#### `private`

Previws for media MAY be shown in "private" rooms without a prompt. A private room is any room where:
- The `m.room.join_rules` state exists.
- The `join_rule` key of this state is `invite`, `knock`, `restricted`, or `knock_restricted`.

If any other `join_rule` is set, or cannot be determined by the client then the assumption MUST be that the
room is public and previews should not be shown. Future join rules may be added to this list, but it's critical
that clients adopt a safety first approach here.

This value is the **default** setting when no account data exists on the user's account.

Note that this setting has no effect for `invite_avatars`. Avatars can only be `off` or `on` for all invites.
Bad actors can easily send a DM to a user (which would pass the `private` check) containing unwanted
content.

### Levels

The account data may exist at both the global and room level. The global setting defines the preference for
all rooms, unless a per-room setting overrides it.

It is also possible for rules to cascade via spaces. A top level space may set a specific rule, and child
rooms may set their own rules. When this is the case, the strictest rule must always be applied.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The below example should probably explore rooms that are in multiple spaces.


For instance, given the following hierarchy of rooms within a space tree:

- Acme Corp <"on">
- Engineering <"private">
- Room A <"off">
- Room B <no-value>
- Support <no-value>
- Room C <"off">
- Room D <"off">
- Room E <no-value>
- Room F

The result would be:
- Room A, Room C and Room D would be "off".
- Room B would be "private".
- Room E would be "on".
- Room F is out of the space, and would default to the user's global rule.

### Notes

It's important here that this account data MUST be configurable by a user.

Homeservers COULD specify a default value ahead of time for the user, by setting a default
value internally for the account data. The user *must* be able to mutate this value.

Not all clients will respect this configuration initially, and many clients will continue to support
their own variant of this setting in the short term.

## Potential issues

TODO: Write this up.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Additionally "on"|"off" may be too coarse for invites.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A setting based on whether you already share a (joined) DM with the inviter could be quite powerful as a trivial form of (cheap) "trust" where checking is just cross-referencing the m.direct global account_data

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This has been added to the alternatives section. The idea is nice albeit I think I'd like to include that in another MSC.

## Alternatives
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe another alternative is to key on sender, not on room type? Ie sharing a private room with a user might be enough to trust their media?

I think at some point someone had explored the idea of "trusted users"...

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

element-hq/element-meta#2780 here I believe, using the mutual rooms API (assuming you're lazyloading members)

It's an interesting idea.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is covered by alternatives now?


This MSC has been iterated on a few times before being published, and several alternatives were considered.

### A well-known field for a global trust policy

This was [originally considered](https://github.com/matrix-org/matrix-spec-proposals/tree/hs/homeserver-content-trust-level)
to change the default policy on a server, so that users on that server would see previews based on their admin's preferences.

This was rejected as it was far too coarse. Expressing a different rule based on the room you are in
was not possible, and wouldn't allow homeservers to store different preferences. Also, confusingly
if the server was a closed registration personal server that happened to federate with the internet
then it would require you to mark your own server as untrusted, which felt wrong in practice.

### Use join rules to determine safety

This is actually part of the proposal under the "private" flag, but this alone wasn't satisfactory for all users. The join
rules have no direct bearing on the content in the room, and bad actors inviting many people to a private room
containing undesirable content would slip under this check.


### Use space membership

Like the above, we could determine that rooms within a trusted space are safe and rooms outside of it are
not. However, this also comes with drawbacks. It would raise the bar for all clients to expose spaces
as a feature in order to make their users safer, and in practice spaces are not fully supported either
by the ecosystem (e.g. Element X) or organisations yet.

### Room state safety flag

Rooms could expose their safeness via a state event, similar to how this proposal does for account data.
However, this would give administrators too much control over your own client experience. It would put the
responsibility on safety on room administrators, who may lack the knowledge or time for proper room configuration.

It also exposes another possible attack where users are invited to a room similar to a phishing attack, and the
room state would override their personal safety settings to deliver undesirable content.

## Security considerations

This field is ultimately held by the homeserver, and a malicious homeserver may expose you to unwanted content. This is
true today, and users should take caution with who they choose to host their account with.

As with all account data fields, the content should be validated.

## Unstable prefix

The field should use `io.element.msc4278.media_preview_config` while the field is unstable.

## Dependencies

None.