Jitsi wrapper developer docs

If you're looking for information on how to set up Jitsi in your Element, see jitsi.md instead.

These docs are for developers wondering how the different conference buttons work within Element. If you're not a developer, you're probably looking for jitsi.md.

Brief introduction to widgets

Widgets are embedded web applications in a room, controlled through state events, and have a url property. They are largely specified by MSC1236 and have extensions proposed under MSC1286.

The url is typically something we shove into an iframe with sandboxing (see AppTile in the react-sdk), though for some widgets special integration can be done. v2 widgets have a data object which helps achieve that special integration, though v1 widgets are best iframed and left alone.

Widgets have a postMessage API they can use to interact with Element, which also allows Element to interact with them. Typically this is most used by the sticker picker (an account-level widget), though widgets like the Jitsi widget will request permissions to get 'stuck' into the room list during a conference.

Widgets can be added with the /addwidget <url> command.

Brief introduction to integration managers

Integration managers (like Scalar and Dimension) are accessible via the 4 squares in the top right of the room and provide a simple UI over top of bridges, bots, and other stuff to plug into a room. They are a separate service to Element and are thus iframed in a dialog as well. They also have a postMessage API they can use to interact with the client to create things like widgets, give permissions to bridges, and generally set everything up for the integration the user is working with.

Integration managers do not currently have a spec associated with them, though efforts are underway in MSC1286.

Widgets configured by integration managers

Integration managers will often "wrap" a widget by using a widget url which points to the integration manager instead of to where the user requested the widget be. For example, a custom widget added in an integration manager for https://matrix.org will end up creating a widget with a URL like https://integrations.example.org?widgetUrl=https%3A%2F%2Fmatrix.org.

The integration manager's wrapper will typically have another iframe to isolate the widget from the client by yet another layer. The wrapper often provides other functionality which might not be available on the embedded site, such as a fullscreen button or the communication layer with the client (all widgets should be talking to the client over postMessage, even if they aren't going to be using the widget APIs).

Widgets added with the /addwidget command will not be wrapped as they are not going through an integration manager. The widgets themselves should also work outside of Element. Widgets currently have a "pop out" button which opens them in a new tab and therefore have no connection back to Riot.

Jitsi widgets from integration managers

Integration managers will create an entire widget event and send it over postMessage for the client to add to the room. This means that the integration manager gets to decide the conference domain, conference name, and other aspects of the widget. As a result, users can end up with a Jitsi widget that does not use the same conference server they specified in their config.json - this is expected.

Some integration managers allow the user to change the conference name while others will generate one for the user.

Jitsi widgets generated by Element itself

When the user clicks on the call buttons by the composer, the integration manager is not involved in the slightest. Instead, Element itself generates a widget event, this time using the config.json parameters, and publishes that to the room. If there's only two people in the room, a plain WebRTC call is made instead of using a widget at all - these are defined in the Matrix specification.

The Jitsi widget created by Element uses a local jitsi.html wrapper (or one hosted by https://app.element.io for desktop users or those on non-https domains) as the widget url. The wrapper has some basic functionality for talking to Element to ensure the required postMessage calls are fulfilled.

Note: Per jitsi.md the preferredDomain can also come from the server's client .well-known data.

The Jitsi wrapper in Element

Whenever Element sees a Jitsi widget, it ditches the url and instead replaces it with its local wrapper, much like what it would do when creating a widget. However, instead of using one from app.element.io, it will use one local to the client instead.

The wrapper is used to provide a consistent experience to users, as well as being faster and less risky to load. The local wrapper URL is populated with the conference information from the original widget (which could be a v1 or v2 widget) so the user joins the right call.

Critically, when the widget URL is reconstructed it does not take into account the config.json's preferredDomain for Jitsi. If it did this, users would end up on different conference servers and therefore different calls entirely.