背景
项目中需要实现私有镜像投屏，而iOS开发中录制全局画面，只能用到系统提供的解决方案：ReplayKit
使用到的技术
1，原生录屏（获取手机端录制的音视频数据）
2，扩展，及与主APP的通讯（解决音视频数据/消息指令在业务逻辑中的传递）
原生录屏（ReplayKit Framework）：
iOS原生录屏发展历史
iOS9: ReplayKit库出现。仅应用内录制，无法录制系统控件（键盘，通知等），无法实时获取原始音视频数据，仅在录制结束后可以获取到编码后的mp4文件。
iOS10:开放Broadcast_Extension，具体是Broadcast Upload Extension和Broadcast Setup UI Extension两个扩展，来实现APP内屏幕录制和实时音视频数据获取。获取的数据可直接通过手机传输到第三方流媒体服务，实现直播的效果。
截屏2025-03-03 15.57.37.png
录制服务可以在启动进程之间插入一个交互页面，用于比如信息鉴权，直播配置等。
iOS11:开放系统级录制，只能由系统控制中心启动。
截屏2025-03-03 16.07.22.png
iOS12:开放标准录制按钮控件，从而实现APP内启动。
小技巧：通过创建标准录制按钮，但不绘制出来，创建点击事件，遍历传递，可以实现自由App内自由唤起录制进程。
全场景录屏：
唤起
//原生录制按钮：
lazy var broadcastBtn : RPSystemBroadcastPickerView = {
let picker = RPSystemBroadcastPickerView.init(frame: CGRect(x: 0, y: 0, width: 40, height: 40))
picker.showsMicrophoneButton = false // 麦克风按钮隐藏
picker.preferredExtension = bundleIdentifier; //仅显示易投屏自身录屏选项
return picker
}()
//模拟点击
for subView in self.systemPicker.subviews {
if let btn = subView as? UIButton {
btn .sendActions(for: .allEvents)
}
}
状态及数据回调
@available(iOS 10.0, *)
open class RPBroadcastSampleHandler : RPBroadcastHandler {
/** @abstract Method is called when the RPBroadcastController startBroadcast method is called from the broadcasting application.
@param setupInfo Dictionary that can be supplied by the UI extension to the sample handler.
*/
open func broadcastStarted(withSetupInfo setupInfo: [String : NSObject]?)
/** @abstract Method is called when the RPBroadcastController pauseBroadcast method is called from the broadcasting application. */
open func broadcastPaused()
/** @abstract Method is called when the RPBroadcastController resumeBroadcast method is called from the broadcasting application. */
open func broadcastResumed()
/** @abstract Method is called when the RPBroadcastController finishBroadcast method is called from the broadcasting application. */
open func broadcastFinished()
/** @abstract Method is called when broadcast is started from Control Center and provides extension information about the first application opened or used during the broadcast.
@param applicationInfo Dictionary that contains information about the first application opened or used buring the broadcast.
*/
@available(iOS 11.2, *)
//监听APP切换。比如只限制直播某些指定app
open func broadcastAnnotated(withApplicationInfo applicationInfo: [AnyHashable : Any])
/** @abstract Method is called as video and audio data become available during a broadcast session and is delivered as CMSampleBuffer objects.
@param sampleBuffer CMSampleBuffer object which contains either video or audio data.
@param sampleBufferType Determine's the type of the sample buffer defined by the RPSampleBufferType enum.
*/
open func processSampleBuffer(_ sampleBuffer: CMSampleBuffer, with sampleBufferType: RPSampleBufferType)
/** @abstract Method that should be called when broadcasting can not proceed due to an error. Calling this method will stop the broadcast and deliver the error back to the broadcasting app through RPBroadcastController's delegate.
@param error NSError object that will be passed back to the broadcasting app through RPBroadcastControllerDelegate's broadcastController:didFinishWithError: method.
*/
open func finishBroadcastWithError(_ error: Error)
}
扩展，及与主App的通讯
iOS的扩展（Extension）
iOS8开始引入的新特性，扩展一般来说是展现在系统级别的 UI 或者是其他应用中的。
是独立的进程，独立的二进制文件，独立的app生命周期，
依赖于主app，无法单独安装。
比如：
Keyboard Extensions（键盘扩展）
Notification Extensions（通知扩展）
Share Extension（共享扩展）
Widget Extension（小部件扩展，灵动岛UI）
Broadcast Extension（广播扩展）
截屏2025-03-04 11.43.09.png
进程间通讯
1，scoket，NERTC使用这种方式将音视频数据传递回主APP然后在主APP中实现编码和上传。
2，AppGroup通知+共享存储空间
设置多个app属于同一AppGroup后，可以使用AppGroup通知中心：CFNotificationCenter。
限制：无法附带userInfo
//发通知：
func sendAPPGroupNotification(notiName:String){
let center : CFNotificationCenter = CFNotificationCenterGetDarwinNotifyCenter()
let identifierRef : CFNotificationName = CFNotificationName(notiName as CFString)
CFNotificationCenterPostNotification(center, identifierRef, nil, nil, true)
}
//接收通知
let center = CFNotificationCenterGetDarwinNotifyCenter()
let observer = UnsafeRawPointer(Unmanaged.passUnretained(self).toOpaque())
let callBack: CFNotificationCallback = {center,observer,name,object,userInfo in
if rawValue == BoardcastExtiension_BoardStart{
...
}
}
CFNotificationCenterAddObserver(center, observer, callBack, BoardcastExtiension_BoardStart as CFString, nil, .deliverImmediately)
//共享存储空间
func saveDataToGroup(data:Data){
let groupUserDefaults = UserDefaults.init(suiteName: groupBundleId)
if let _ = data {
let dic = ["type" : "KKmedia","infoData":data!] as [String : Any]
groupUserDefaults?.setValue(dic, forKey: UserDefaults_Group_LastLinkTVInfo)
}
}
限制：无法附带userInfo
后台常驻：
子进程录制期间是常驻的，但是主进程会在后台挂起，为了保持进程间通讯，需要实现后台常驻。
正常app可以申请延长后台活跃时间（目前最多延长到15秒左右）：
//以下两个方法必须成对出现。
//申请后台时间
func registerBackgroundTask() {
if self.needBackGroundTask{
print("registerBackgroundTask")
backgroundTask = UIApplication.shared.beginBackgroundTask { [weak self] in
print("iOS has signaled time has expired")
self?.endBackgroundTaskIfActive()
}
}
}
//后台活跃时间结束时的回收处理。
func endBackgroundTaskIfActive() {
let isBackgroundTaskActive = backgroundTask != .invalid
if isBackgroundTaskActive {
print("Background task ended.")
if self.backgroundTask != nil{
UIApplication.shared.endBackgroundTask(backgroundTask!)
}
backgroundTask = .invalid
}
}
指定场景可以无限延长后台常驻时间：导航，voip，音乐播放，画中画等。
截屏2025-03-17 16.46.31.png
通过后台播放静音文件，加循环申请后台时间延长，实现无限延长的后台常驻。
审核相关：app提交审核时，需要证明app确实有后台场景，目前是提交了一份录像：电视播放音乐，传屏到手机，然后手机进入后台后持续播放音乐。
代码：
//后台播放静音文件：
class BackGroundPlayer : NSObject,AVAudioPlayerDelegate
{
static let share = BackGroundPlayer()
var audioPlayer : AVAudioPlayer?
func start(){
self.setAudioPlaySession()
self.playSound()
}
func setAudioPlaySession(){
let audioSession = AVAudioSession.sharedInstance()
do{
try audioSession.setCategory(.playback, options: .mixWithOthers)
}catch{
}
}
func playSound(){
if self.audioPlayer == nil {
guard let filePath = Bundle.main.path(forResource: "RunInBackground", ofType: "mp3") else{
return
}
let url = URL.init(fileURLWithPath: filePath)
do{
try self.audioPlayer = AVAudioPlayer.init(contentsOf: url, fileTypeHint: nil)
self.audioPlayer?.volume = 0.0
}catch{
}
}
self.audioPlayer?.play()
}
}
//AppDelegate.swift
//定义开启后台常驻开关
var needBackGroundTask = false
//进入后台，启动循环申请
func applicationDidEnterBackground(_ application: UIApplication) {
print("applicationDidEnterBackground")
if self.needBackGroundTask{
self.backGroundTimer = Timer.scheduledTimer(timeInterval: 5, target: self, selector: #selector(startLoopBackGroundTask), userInfo: nil, repeats: true)
RunLoop.current.add(self.backGroundTimer!, forMode: .common)
self.backGroundTimer?.fire()
self.registerBackgroundTask()
}
}
//回到前台
func applicationWillEnterForeground(_ application: UIApplication) {
print("applicationWillEnterForeground")
self.backGroundTimer?.invalidate()
self.backGroundTimer = nil
self.endBackgroundTaskIfActive()
}
//主进程结束时，强制关闭子进程
func applicationWillTerminate(_ application: UIApplication) {
print("applicationWillTerminate")
sendBoardCastStop(error: nil)
}
//启动后台常驻
@objc func startLoopBackGroundTask(){
print("startLoopBackGroundTask")
//判断后台剩余时间小于19秒则再次重新获取后台时间延长
if UIApplication.shared.backgroundTimeRemaining < 19{
BackGroundPlayer.share.start()
self.endBackgroundTaskIfActive()
self.registerBackgroundTask()
}
}
//申请后台时间
func registerBackgroundTask() {
if self.needBackGroundTask{
print("registerBackgroundTask")
backgroundTask = UIApplication.shared.beginBackgroundTask { [weak self] in
print("iOS has signaled time has expired")
self?.endBackgroundTaskIfActive()
}
}
}
//结束后台常驻时的处理
func endBackgroundTaskIfActive() {
let isBackgroundTaskActive = backgroundTask != .invalid
if isBackgroundTaskActive {
print("Background task ended.")
if self.backgroundTask != nil{
UIApplication.shared.endBackgroundTask(backgroundTask!)
}
backgroundTask = .invalid
}
}
-----以上所有技术问题都已解决-------
传屏涉及的iOS的特性（或者说限制，边界）：
1，何时算是开始传屏（3秒问题）
2，error提示（APP内/APP外），原生录屏crash，如何触发弹框，系统弹框样式：
截屏2025-03-17 17.23.39.png
3，横竖屏切换问题：录屏返回的帧画面在开始录屏时确定，同时会在返回的录屏数据中返回画面朝向（这个朝向严格来说是设备的朝向。）目前是在扩展中根据朝向参数，来旋转帧画面后，拿到新的帧画面输入到编码器。
4，50M限制：目前场景无美颜，AI识别等操作，正常不会超出50M限制。
