Video Experience

If you have a Plex setup, either with Sonarr or Radarr or just a folder with movies and TV shows, you may notice that it's a bit of a challenge to watch content (i.e., decode different types of media) on various operating systems (e.g., Windows, macOS, iOS, Android, TV OS). This can become more challenging with an HDR setup.

Media players

I've listed different solutions for playing Plex videos on desktop operating systems with their pros and cons:

Watch together experience

It becomes more of a problem when you want to share or watch videos in your Plex library with your friends.

There are 2 main ways to do this:

1. Use Plex's built-in watch together, your friends would have to create a Plex account and go through the media player setups on their devices. The built-in watch together also lacks features like chats, room resumes, media switching, room sharing, etc. It's not a great experience.
2. Stream the video to your friend. This works, but the video will be re-encoded by your streaming software. The video quality varies depending on your network setup and your friends'. I do not think there's a platform that supports HDR streaming either.

3 years ago, I made a Python script to sync playback status across different players. However, it's unreliable to retrieve the status across different players and control them effectively. Additionally, this doesn’t address the challenge of playing videos on various operating systems.

Sparkle

Since modern browsers support most of the codecs and HDR. For instance, you could play back HDR videos (AV1, H.265, H.264) on Chromium and (H.265, H.264) Safari (newer models also support AV1). Firefox also supports AV1.

This means ideally, a solution could be created to support smooth playback of any media in your library on a modern browser both for yourself and your friends.

I decided to create a solution that would solve all aforementioned challenges.

This comes with 3 parts:

1. Video pipelines

A server that automatically prepares the video files for streaming. It does the following:

1. Encode the video files to AV1, HEVC, H.264 codecs
2. Generate thumbnails (video storyboards for preview) and posters
3. Extract subtitles and audio tracks
4. Locate and extract nfo files
5. Extract dominant theme color from the poster
6. Extract and populate chapter timestamps & titles

2. WebSocket backend

A room-based pub/sub backend written in Go that supports real-time syncing like chat, profile picture, pause/play, seek, and media switch.

3. Fullstack web application

Frontend and APIs written in SvelteKit:

Video playback

• pause/play, seek real-time syncing with the server
• HDR with automatic AV1, HEVC, H.264 codec switching (based on browser support)
• All main stream subtitles & language selection (note: iOS only supports WebVTT in fullscreen):
  • SSA/ASS
  • WebVTT
  • SUP (the image subtitle format)
  • SRT
• Thumbnail (Video Storyboard/Preview) | Poster (Video Cover)
• Video chapter display with timestamps

Media selection

• Multiple audio selection
• Media selection with season and episode support

Room/Chat syncing

• Synced media switching for all players in the room
• chat, profile picture, and name real-time syncing
• Automatic room creation
• Auto Reconnect
• In-background (i.e., tabbed out) notifications
• Chat auto-hide based on input/fullscreen status

Link sharing

• HTML embed headers with dominant color based on show/episode
• Full oembed support with thumbnail and episode information

Live demo

I've integrated the solution with my entire Plex library. However, sharing a full demo would be challenging due to the nature of the content. I've created a demo with a few video sample files: Sparkle.

You could simply select a media file, share the room link with a friend, and play around with the features. Room creation is automatic.

Discord App Support

Most of my friends who watch videos together are on Discord. I decided to integrate the solution with Discord to provide a seamless experience.

I designed the site to check Discord OAuth2 for username and profile picture syncing. Channel id will be used as the room id. All functionalities mentioned above are supported in Discord activity.

This allows us to watch videos together in Discord using the same Sparkle stack by launching a Discord application.

Activity setup

• Create a new application in Discord Developer Portal
• Enable activities
• Add the site url as the redirect url and root url
• Add Discord users to Application Testers (they don’t need to accept invites)
• Users will be prompted to authorize with identify and rpc.activities.write scope on joining the activity

Discord status

I've also integrated with Discord status. The site will display the current room status in the user's Discord status. Sharing a link also creates a full embed thanks to oembed.json.