1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348
|
// Copyright 2014 The Chromium Authors
// Use of this source code is governed by a BSD-style license that can be
// found in the LICENSE file.
#ifndef CONTENT_BROWSER_MEDIA_MEDIA_WEB_CONTENTS_OBSERVER_H_
#define CONTENT_BROWSER_MEDIA_MEDIA_WEB_CONTENTS_OBSERVER_H_
#include <stdint.h>
#include <map>
#include <memory>
#include <optional>
#include "base/containers/flat_map.h"
#include "base/memory/raw_ptr.h"
#include "base/memory/weak_ptr.h"
#include "build/build_config.h"
#include "content/browser/media/audio_stream_monitor.h"
#include "content/browser/media/media_devices_util.h"
#include "content/browser/media/media_power_experiment_manager.h"
#include "content/browser/media/session/media_session_controllers_manager.h"
#include "content/common/content_export.h"
#include "content/public/browser/global_routing_id.h"
#include "content/public/browser/media_player_id.h"
#include "content/public/browser/render_frame_host.h"
#include "content/public/browser/web_contents_observer.h"
#include "media/base/use_after_free_checker.h"
#include "media/mojo/mojom/media_player.mojom.h"
#include "mojo/public/cpp/bindings/associated_receiver.h"
#include "mojo/public/cpp/bindings/associated_receiver_set.h"
#include "mojo/public/cpp/bindings/associated_remote.h"
#include "mojo/public/cpp/bindings/pending_associated_receiver.h"
#include "mojo/public/cpp/bindings/pending_associated_remote.h"
#include "mojo/public/cpp/bindings/remote.h"
#include "services/device/public/mojom/wake_lock.mojom.h"
#if BUILDFLAG(IS_ANDROID)
#include "ui/android/view_android.h"
#endif // BUILDFLAG(IS_ANDROID)
namespace blink {
enum class WebFullscreenVideoStatus;
} // namespace blink
namespace media {
enum class MediaContentType;
} // namespace media
namespace media_session {
struct MediaPosition;
} // namespace media_session
namespace gfx {
class Size;
} // namespace gfx
namespace content {
class AudibleMetrics;
class WebContentsImpl;
// This class manages all RenderFrame based media related managers at the
// browser side. It receives IPC messages from media RenderFrameObservers and
// forwards them to the corresponding managers. The managers are responsible
// for sending IPCs back to the RenderFrameObservers at the render side.
class CONTENT_EXPORT MediaWebContentsObserver
: public WebContentsObserver,
public media::mojom::MediaPlayerObserverClient {
public:
explicit MediaWebContentsObserver(WebContentsImpl* web_contents);
MediaWebContentsObserver(const MediaWebContentsObserver&) = delete;
MediaWebContentsObserver& operator=(const MediaWebContentsObserver&) = delete;
~MediaWebContentsObserver() override;
// Called by WebContentsImpl when the audible state may have changed.
void MaybeUpdateAudibleState();
// Called by WebContentsImpl to know if an active player is effectively
// fullscreen. That means that the video is either fullscreen or it is the
// content of a fullscreen page (in other words, a fullscreen video with
// custom controls).
// It should only be called while the WebContents is fullscreen.
bool HasActiveEffectivelyFullscreenVideo() const;
// Called by WebContentsImpl to know if Picture-in-Picture can be triggered
// for the current active effectively fullscreen player.
// It should only be called while the WebContents is fullscreen.
bool IsPictureInPictureAllowedForFullscreenVideo() const;
// Gets the MediaPlayerId of the fullscreen video if it exists.
const std::optional<MediaPlayerId>& GetFullscreenVideoMediaPlayerId() const;
// WebContentsObserver implementation.
void WebContentsDestroyed() override;
void RenderFrameDeleted(RenderFrameHost* render_frame_host) override;
void MediaPictureInPictureChanged(bool is_picture_in_picture) override;
void DidUpdateAudioMutingState(bool muted) override;
void DidStartNavigation(NavigationHandle* navigation_handle) override;
void RenderFrameHostChanged(RenderFrameHost* old_host,
RenderFrameHost* new_host) override;
// MediaPlayerObserverClient implementation.
void GetHasPlayedBefore(GetHasPlayedBeforeCallback callback) override;
void BindMediaPlayerObserverClient(
mojo::PendingReceiver<media::mojom::MediaPlayerObserverClient>
pending_receiver);
// TODO(zqzhang): this method is temporarily in MediaWebContentsObserver as
// the effectively fullscreen video code is also here. We need to consider
// merging the logic of effectively fullscreen, hiding media controls and
// fullscreening video element to the same place.
void RequestPersistentVideo(bool value);
// Returns the number of active players with video content.
int GetCurrentlyPlayingVideoCount() const;
// Returns whether or not the given player id is active.
bool IsPlayerActive(const MediaPlayerId& player_id) const;
bool has_audio_wake_lock_for_testing() const {
return has_audio_wake_lock_for_testing_;
}
void SetAudibleMetricsForTest(AudibleMetrics* audible_metrics) {
audible_metrics_ = audible_metrics;
}
// Returns whether or not to be able to use the MediaPlayer mojo interface.
bool IsMediaPlayerRemoteAvailable(const MediaPlayerId& player_id);
// Return an already bound mojo Remote for the MediaPlayer mojo interface. It
// is an error to call this method if no MediaPlayer with |player_id| exists.
mojo::AssociatedRemote<media::mojom::MediaPlayer>& GetMediaPlayerRemote(
const MediaPlayerId& player_id);
// Creates a new MediaPlayerObserverHostImpl associated to |player_id| if
// needed, and then passes |player_receiver| to it to establish a
// communication channel.
void BindMediaPlayerHost(
GlobalRenderFrameHostId frame_routing_id,
mojo::PendingAssociatedReceiver<media::mojom::MediaPlayerHost>
player_receiver);
// Called by the WebContents when a tab has been closed but may still be
// available for "undo" -- indicates that all media players (even audio only
// players typically allowed background audio) bound to this WebContents must
// be suspended.
void SuspendAllMediaPlayers();
protected:
MediaSessionControllersManager* session_controllers_manager() {
return session_controllers_manager_.get();
}
private:
class PlayerInfo;
using PlayerInfoMap =
base::flat_map<MediaPlayerId, std::unique_ptr<PlayerInfo>>;
// Helper class providing a per-RenderFrame object implementing the only
// method of the media::mojom::MediaPlayerHost mojo interface, to provide the
// renderer process with a way to notify the browser when a new MediaPlayer
// has been created, so that a communication channel can be established.
class MediaPlayerHostImpl : public media::mojom::MediaPlayerHost {
public:
MediaPlayerHostImpl(GlobalRenderFrameHostId frame_routing_id,
MediaWebContentsObserver* media_web_contents_observer);
~MediaPlayerHostImpl() override;
// Used to bind receivers via the BrowserInterfaceBroker.
void AddMediaPlayerHostReceiver(
mojo::PendingAssociatedReceiver<media::mojom::MediaPlayerHost>
receiver);
// media::mojom::MediaPlayerHost implementation.
void OnMediaPlayerAdded(
mojo::PendingAssociatedRemote<media::mojom::MediaPlayer> media_player,
mojo::PendingAssociatedReceiver<media::mojom::MediaPlayerObserver>
media_player_observer,
int32_t player_id) override;
private:
GlobalRenderFrameHostId frame_routing_id_;
raw_ptr<MediaWebContentsObserver> media_web_contents_observer_;
mojo::AssociatedReceiverSet<media::mojom::MediaPlayerHost> receivers_;
};
// Helper class providing a per-MediaPlayerId object implementing the
// media::mojom::MediaPlayerObserver mojo interface.
class MediaPlayerObserverHostImpl : public media::mojom::MediaPlayerObserver {
public:
MediaPlayerObserverHostImpl(
const MediaPlayerId& media_player_id,
MediaWebContentsObserver* media_web_contents_observer);
~MediaPlayerObserverHostImpl() override;
// Used to bind the receiver via the BrowserInterfaceBroker.
void BindMediaPlayerObserverReceiver(
mojo::PendingAssociatedReceiver<media::mojom::MediaPlayerObserver>
media_player_observer);
// media::mojom::MediaPlayerObserver implementation.
void OnMediaPlaying() override;
void OnMediaPaused(bool stream_ended) override;
void OnMutedStatusChanged(bool muted) override;
void OnMediaMetadataChanged(
bool has_audio,
bool has_video,
media::MediaContentType media_content_type) override;
void OnMediaPositionStateChanged(
const media_session::MediaPosition& media_position) override;
void OnMediaEffectivelyFullscreenChanged(
blink::WebFullscreenVideoStatus status) override;
void OnMediaSizeChanged(const ::gfx::Size& size) override;
void OnPictureInPictureAvailabilityChanged(bool available) override;
void OnAudioOutputSinkChanged(const std::string& hashed_device_id) override;
void OnUseAudioServiceChanged(bool uses_audio_service) override;
void OnAudioOutputSinkChangingDisabled() override;
void OnRemotePlaybackMetadataChange(
media_session::mojom::RemotePlaybackMetadataPtr
remote_playback_metadata) override;
void OnVideoVisibilityChanged(bool meets_visibility_threshold) override;
private:
PlayerInfo* GetPlayerInfo();
void NotifyAudioStreamMonitorIfNeeded();
void OnReceivedMediaDeviceSalt(
const std::string& hashed_device_id,
const content::MediaDeviceSaltAndOrigin& salt_and_origin);
void OnReceivedTranslatedDeviceId(
const std::optional<std::string>& translated_id);
const MediaPlayerId media_player_id_;
const raw_ptr<MediaWebContentsObserver> media_web_contents_observer_;
mojo::AssociatedReceiver<media::mojom::MediaPlayerObserver>
media_player_observer_receiver_{this};
// Helps monitor audio stream when not using AudioService.
bool uses_audio_service_ = true;
std::unique_ptr<AudioStreamMonitor::AudibleClientRegistration>
audio_client_registration_;
base::WeakPtrFactory<MediaPlayerObserverHostImpl> weak_factory_{this};
};
using MediaPlayerHostImplMap =
base::flat_map<GlobalRenderFrameHostId,
std::unique_ptr<MediaPlayerHostImpl>>;
using MediaPlayerObserverHostImplMap =
base::flat_map<MediaPlayerId,
std::unique_ptr<MediaPlayerObserverHostImpl>>;
using MediaPlayerRemotesMap =
base::flat_map<MediaPlayerId,
mojo::AssociatedRemote<media::mojom::MediaPlayer>>;
// Communicates with the MediaSessionControllersManager to find or create (if
// needed) a MediaSessionController identified by |player_id|, in order to
// bind its mojo remote for media::mojom::MediaPlayer.
void OnMediaPlayerAdded(
mojo::PendingAssociatedRemote<media::mojom::MediaPlayer> player_remote,
mojo::PendingAssociatedReceiver<media::mojom::MediaPlayerObserver>
media_player_observer,
MediaPlayerId player_id);
// Returns the PlayerInfo associated with |id|, or nullptr if no such
// PlayerInfo exists.
PlayerInfo* GetPlayerInfo(const MediaPlayerId& id) const;
void OnMediaMetadataChanged(const MediaPlayerId& player_id,
bool has_video,
bool has_audio,
media::MediaContentType media_content_type);
void OnMediaEffectivelyFullscreenChanged(
const MediaPlayerId& player_id,
blink::WebFullscreenVideoStatus fullscreen_status);
void OnMediaPlaying();
void OnAudioOutputSinkChangedWithRawDeviceId(
const MediaPlayerId& player_id,
const std::string& raw_device_id);
void OnRemotePlaybackMetadataChange(
const MediaPlayerId& player_id,
media_session::mojom::RemotePlaybackMetadataPtr remote_playback_metadata);
// Used to notify when the renderer -> browser mojo connection via the
// interface media::mojom::MediaPlayerObserver gets disconnected.
void OnMediaPlayerObserverDisconnected(const MediaPlayerId& player_id);
device::mojom::WakeLock* GetAudioWakeLock();
// WakeLock related methods for audio and video.
void LockAudio();
void CancelAudioLock();
void UpdateVideoLock();
// Convenience method that casts web_contents() to a WebContentsImpl*.
WebContentsImpl* web_contents_impl() const;
// Notify |id| about |is_starting|. Note that |id| might no longer be in the
// active players list, which is fine.
void OnExperimentStateChanged(MediaPlayerId id, bool is_starting);
// Return a weak pointer to |this| that's local to |render_frame_host|, in the
// sense that we can cancel all of the ptrs to one frame without cancelling
// pointers for any of the others.
base::WeakPtr<MediaWebContentsObserver> GetWeakPtrForFrame(
RenderFrameHost* render_frame_host);
// Helper class for recording audible metrics.
raw_ptr<AudibleMetrics> audible_metrics_;
// A boolean indicating whether media has played before.
bool has_played_before_ = false;
mojo::ReceiverSet<media::mojom::MediaPlayerObserverClient> receivers_;
// Tracking variables and associated wake locks for media playback.
PlayerInfoMap player_info_map_;
mojo::Remote<device::mojom::WakeLock> audio_wake_lock_;
std::optional<MediaPlayerId> fullscreen_player_;
std::optional<bool> picture_in_picture_allowed_in_fullscreen_;
bool has_audio_wake_lock_for_testing_ = false;
std::unique_ptr<MediaSessionControllersManager> session_controllers_manager_;
raw_ptr<MediaPowerExperimentManager> power_experiment_manager_ = nullptr;
std::map<RenderFrameHost*,
std::unique_ptr<base::WeakPtrFactory<MediaWebContentsObserver>>>
per_frame_factory_;
media::UseAfterFreeChecker use_after_free_checker_;
MediaPlayerHostImplMap media_player_hosts_;
MediaPlayerObserverHostImplMap media_player_observer_hosts_;
// Map of remote endpoints for the media::mojom::MediaPlayer mojo interface,
// indexed by MediaPlayerId.
MediaPlayerRemotesMap media_player_remotes_;
};
} // namespace content
#endif // CONTENT_BROWSER_MEDIA_MEDIA_WEB_CONTENTS_OBSERVER_H_
|