Skip to content

Supports audio sharing#181

Closed
strvert wants to merge 7 commits intoscreego:masterfrom
strvert:support-audio
Closed

Supports audio sharing#181
strvert wants to merge 7 commits intoscreego:masterfrom
strvert:support-audio

Conversation

@strvert
Copy link
Copy Markdown

@strvert strvert commented Jun 9, 2024

I understand that Screego aims to remain extremely simple, and that audio support was previously closed in issue #102. However, in my daily work, I find it inconvenient not to be able to share audio when I need to demonstrate code and its results to colleagues. As a game programmer, a lot of sound is produced as a result of my code. Not being able to convey this to my colleagues is a significant drawback.

Considering the context of issue #102, I have implemented a slightly different concept, which I will explain below.

Audio Transmission Settings

image

In the browser menu, users can choose whether to share audio. If enabled, both video and audio tracks will be sent; if disabled, only the video track will be sent. In browsers that do not support audio sharing, such as Firefox, this menu will not be displayed, and only the video track will always be sent.

Audio Listening Settings

First, audio can only be listened to under the following conditions:

  1. It is not the hostStream.
  2. It is the selectedStream.
  3. The "Hear Audio" button is enabled.

The reason for 1 is straightforward—nobody wants to hear their own screen's audio twice.

For 2, the concept of this implementation is the reason. When receiving multiple screens, I couldn't think of a useful scenario where overlapping audio is helpful. Also, having many mute buttons displayed would not be simple. Therefore, by limiting the audio track used for playback to the selected stream, users can choose which audio to hear.

For 3, this is the only additional button in this implementation. With just 2, you can choose which audio to hear but cannot choose not to hear any audio at all. So, I added a button to the control menu to toggle the audio on and off. The default state is Mute. If the selected stream is your own screen or does not include an audio track, the button will automatically be disabled.

image
image
image

Of course, even in browsers that do not support audio sharing, such as Firefox, you can still receive audio.

I believe this approach maintains simplicity.

@strvert strvert changed the title Support audio Supports audio sharing Jun 12, 2024
Copy link
Copy Markdown
Member

@jmattheis jmattheis left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I still think this feature doesn't entirely fit within the project's scope. However, it can be argued that system audio is a part of screen sharing and thus enhances the experience of using screego.

@strvert
Copy link
Copy Markdown
Author

strvert commented Jun 16, 2024

I expelicitly set echoCancellation and noiseSuppression to false to disable audio effects applied by the browser.
I combined the MediaStreams into one and added the AudioTrack and VideoTrack to it.
Additionally, the audio icon will not be displayed when there is no AudioTrack.

@strvert
Copy link
Copy Markdown
Author

strvert commented Jun 16, 2024

For consistency with other features, I added the hotkey 'a' for audio. Along with this, an issue arose from the timing of initilizing the audio element reference, so I replaced it with the more appropriate useRef.

}
}

onTrack(stream);
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This can be moved inside the if (event.track.kind === 'video), then we don't have to adjust the onTrack setState implementation at all.

)}

{audioStream && (
<audio
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The <video> element should be able to play audio, we should use this instead of creating another audio element. the muted property should be used to control if the audio should be played or not.

},
[state.clientStreams, selectedStream]
);
useHotkeys('a', toggleAudio, [playingAudio]);
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the hotkey should be m for mute.

videoElement.srcObject = stream;
if (videoElement && videoStream) {
videoElement.srcObject = videoStream;
videoElement.play().catch((e) => console.log('Could not play main video', e));
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The video audio can be muted when in fullscreen via native controls, the playingAudio state in screego should reflect that. PlayingAudio should be updated via volume events from the video e.g.:

            videoElement.onvolumechange = () => setPlayingAudio(videoElement.muted)

@Mondrethos
Copy link
Copy Markdown

Any chance this will continue? @jmattheis @strvert

@beyondgrayzone
Copy link
Copy Markdown

If anyone is interested, latest changes from this PR can be found in this release https://github.com/beyondgrayzone/screego-server/releases/tag/v1.11.3.

@Bristopher
Copy link
Copy Markdown

If anyone is interested, latest changes from this PR can be found in this release https://github.com/beyondgrayzone/screego-server/releases/tag/v1.11.3.

Thank you!!

@ajmandourah
Copy link
Copy Markdown

If anyone is interested, latest changes from this PR can be found in this release https://github.com/beyondgrayzone/screego-server/releases/tag/v1.11.3.

would you mind making a docker image out of it. Thanks

@beyondgrayzone
Copy link
Copy Markdown

If anyone is interested, latest changes from this PR can be found in this release https://github.com/beyondgrayzone/screego-server/releases/tag/v1.11.3.

would you mind making a docker image out of it. Thanks

https://hub.docker.com/r/beyondgrayzone/screego-server/tags 💪

@jmattheis
Copy link
Copy Markdown
Member

This is now supported with v1.12.0.

@jmattheis jmattheis closed this May 8, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants