版本记录

前言

GPUImage是直接利用显卡实现视频或者图像处理的技术。感兴趣可以看上面几篇文章。
1. GPUImage解析（一） —— 基本概览（一）
2. GPUImage解析（二） —— 基本概览（二）
3. GPUImage解析（三） —— 基本概览（三）
4. GPUImage解析（四） —— 安装方法及框架介绍

框架中的几个基类

该框架其实可以分为两个部分，一部分就是基类，另外一部分就是滤镜，这一篇我们就说一下这个框架的基类部分。

// Base classes

#import "GPUImageContext.h"
#import "GPUImageOutput.h"
#import "GPUImageView.h"
#import "GPUImageVideoCamera.h"
#import "GPUImageStillCamera.h"
#import "GPUImageMovie.h"
#import "GPUImagePicture.h"
#import "GPUImageRawDataInput.h"
#import "GPUImageRawDataOutput.h"
#import "GPUImageMovieWriter.h"
#import "GPUImageFilterPipeline.h"
#import "GPUImageTextureOutput.h"
#import "GPUImageFilterGroup.h"
#import "GPUImageTextureInput.h"
#import "GPUImageUIElement.h"
#import "GPUImageBuffer.h"
#import "GPUImageFramebuffer.h"
#import "GPUImageFramebufferCache.h"

基类详细分析

下面我们就详细的分析下这几个基类。

1. GPUImageContext

• 继承与属性

@interface GPUImageContext : NSObject

@property(readonly, nonatomic) dispatch_queue_t contextQueue;
@property(readwrite, retain, nonatomic) GLProgram *currentShaderProgram;
@property(readonly, retain, nonatomic) EAGLContext *context;
@property(readonly) CVOpenGLESTextureCacheRef coreVideoTextureCache;
@property(readonly) GPUImageFramebufferCache *framebufferCache;

• 作用：GPUImageContext是GPUImage对OpenGL ES上下文的封装，添加了GPUImage相关的上下文，比如说Program的使用缓存，处理队列，CV纹理缓存等。

几个属性

• contextQueue 统一处理队列
• currentShaderProgram 正在使用的program
• context OpenGL ES的上下文
• coreVideoTextureCache CV纹理缓存
• framebufferCache GPUImageBuffer缓存
• shaderProgramCache Program的缓存
• shaderProgramUsageHistory Program的使用历史

几个方法

• useAsCurrentContext()在useAsCurrentContext设置当前上下文的时候，会先判断上下文是否是当前context，不是再设置（为了避免上下文切换的性能消耗，即使设置的上下文是同一个上下文也会消耗性能）

• sizeThatFitsWithinATextureForSize()会调整纹理大小，如果超过最大的纹理，会调整为不超过最大的纹理宽高。

• (GLProgram*)programForVertexShaderString:fragmentShaderString:
  shaderProgramCache 是program的缓存，由顶点shader和片元shader字符串拼接起来做key。

• (void)useSharegroup:(EAGLSharegroup *)sharegroup;
  EAGLSharegroup类管理一个或者多个EAGLContext的OpenGLES资源；这个是一个封闭的类，没有开发者API。负责管理纹理缓存、顶点缓存、帧缓存、颜色缓存。（textures, buffers, framebuffers, and render buffers）

• (EAGLContext *)context;返回OpenGL ES2.0的上下文，同时设置glDisable(GL_DEPTH_TEST);，图像处理管道默认不允许使用深度缓存。

2. GPUImageOutput

• 继承与属性

/** GPUImage's base source object
 
 Images or frames of video are uploaded from source objects, which are subclasses of GPUImageOutput. These include:
 
 - GPUImageVideoCamera (for live video from an iOS camera) 
 - GPUImageStillCamera (for taking photos with the camera)
 - GPUImagePicture (for still images)
 - GPUImageMovie (for movies)
 
 Source objects upload still image frames to OpenGL ES as textures, then hand those textures off to the next objects in the processing chain.
 */
@interface GPUImageOutput : NSObject
{
    GPUImageFramebuffer *outputFramebuffer;
    
    NSMutableArray *targets, *targetTextureIndices;
    
    CGSize inputTextureSize, cachedMaximumOutputSize, forcedMaximumSize;
    
    BOOL overrideInputSize;
    
    BOOL allTargetsWantMonochromeData;
    BOOL usingNextFrameForImageCapture;
}

• 作用：GPUImageOutput类将静态图像纹理上传到OpenGL ES中，然后使用这些纹理去处理进程链中的下一个对象。它的子类可以获得滤镜处理后的图片功能。

3. GPUImageView

• 继承与属性

/**
 UIView subclass to use as an endpoint for displaying GPUImage outputs
 */
@interface GPUImageView : UIView <GPUImageInput>
{
    GPUImageRotationMode inputRotation;
}

/** The fill mode dictates how images are fit in the view, with the default being kGPUImageFillModePreserveAspectRatio
 */
@property(readwrite, nonatomic) GPUImageFillModeType fillMode;

/** This calculates the current display size, in pixels, taking into account Retina scaling factors
 */
@property(readonly, nonatomic) CGSize sizeInPixels;

@property(nonatomic) BOOL enabled;

/** Handling fill mode
 
 @param redComponent Red component for background color
 @param greenComponent Green component for background color
 @param blueComponent Blue component for background color
 @param alphaComponent Alpha component for background color
 */
- (void)setBackgroundColorRed:(GLfloat)redComponent green:(GLfloat)greenComponent blue:(GLfloat)blueComponent alpha:(GLfloat)alphaComponent;

- (void)setCurrentlyReceivingMonochromeInput:(BOOL)newValue;

@end

• 图像视图

4. GPUImageVideoCamera

• 继承与属性

/**
 A GPUImageOutput that provides frames from either camera
*/
@interface GPUImageVideoCamera : GPUImageOutput <AVCaptureVideoDataOutputSampleBufferDelegate, AVCaptureAudioDataOutputSampleBufferDelegate>
{
    NSUInteger numberOfFramesCaptured;
    CGFloat totalFrameTimeDuringCapture;
    
    AVCaptureSession *_captureSession;
    AVCaptureDevice *_inputCamera;
    AVCaptureDevice *_microphone;
    AVCaptureDeviceInput *videoInput;
    AVCaptureVideoDataOutput *videoOutput;

    BOOL capturePaused;
    GPUImageRotationMode outputRotation, internalRotation;
    dispatch_semaphore_t frameRenderingSemaphore;
        
    BOOL captureAsYUV;
    GLuint luminanceTexture, chrominanceTexture;

    __unsafe_unretained id<GPUImageVideoCameraDelegate> _delegate;
}

/// The AVCaptureSession used to capture from the camera
@property(readonly, retain, nonatomic) AVCaptureSession *captureSession;

/// This enables the capture session preset to be changed on the fly
@property (readwrite, nonatomic, copy) NSString *captureSessionPreset;

/// This sets the frame rate of the camera (iOS 5 and above only)
/**
 Setting this to 0 or below will set the frame rate back to the default setting for a particular preset.
 */
@property (readwrite) int32_t frameRate;

/// Easy way to tell which cameras are present on device
@property (readonly, getter = isFrontFacingCameraPresent) BOOL frontFacingCameraPresent;
@property (readonly, getter = isBackFacingCameraPresent) BOOL backFacingCameraPresent;

/// This enables the benchmarking mode, which logs out instantaneous and average frame times to the console
@property(readwrite, nonatomic) BOOL runBenchmark;

/// Use this property to manage camera settings. Focus point, exposure point, etc.
@property(readonly) AVCaptureDevice *inputCamera;

/// This determines the rotation applied to the output image, based on the source material
@property(readwrite, nonatomic) UIInterfaceOrientation outputImageOrientation;

/// These properties determine whether or not the two camera orientations should be mirrored. By default, both are NO.
@property(readwrite, nonatomic) BOOL horizontallyMirrorFrontFacingCamera, horizontallyMirrorRearFacingCamera;

@property(nonatomic, assign) id<GPUImageVideoCameraDelegate> delegate;

• 作用：GPUImageVideoCamera是GPUImageOutput的子类，提供来自摄像头的图像数据作为源数据，一般是响应链的源头。GPUImage使用AVFoundation框架来获取视频。
  AVCaptureSession类从AV输入设备的采集数据到制定的输出。为了实现实时的图像捕获，要实现AVCaptureSession类，添加合适的输入（AVCaptureDeviceInput）和输出（比如 AVCaptureMovieFileOutput）调用startRunning开始输入到输出的数据流，调用stopRunning停止数据流。需要注意的是startRunning函数会花费一定的时间，所以不能在主线程（UI线程）调用，防止卡顿。

5. GPUImageStillCamera

• 继承与属性

@interface GPUImageStillCamera : GPUImageVideoCamera

/** The JPEG compression quality to use when capturing a photo as a JPEG.
 */
@property CGFloat jpegCompressionQuality;

// Only reliably set inside the context of the completion handler of one of the capture methods
@property (readonly) NSDictionary *currentCaptureMetadata;

• 作用：GPUImage中GPUImageStillCamera可以调用系统相机，并实现实时滤镜，GPUImageStillCamera继承自GPUImageVideoCamera类，添加了捕获照片的功能。

使用步骤

• 创建预览View 即必须的GPUImageView
• 创建滤镜
• 创建Camera 即我们要用到的GPUImageStillCamera
• addTarget 并开始处理startCameraCapture
• 回调数据、写入相册

6. GPUImageMovie

• 继承与属性

/** Source object for filtering movies
 */
@interface GPUImageMovie : GPUImageOutput

@property (readwrite, retain) AVAsset *asset;
@property (readwrite, retain) AVPlayerItem *playerItem;
@property(readwrite, retain) NSURL *url;

/** This enables the benchmarking mode, which logs out instantaneous and average frame times to the console
 */
@property(readwrite, nonatomic) BOOL runBenchmark;

/** This determines whether to play back a movie as fast as the frames can be processed, or if the original speed of the movie should be respected. Defaults to NO.
 */
@property(readwrite, nonatomic) BOOL playAtActualSpeed;

/** This determines whether the video should repeat (loop) at the end and restart from the beginning. Defaults to NO.
 */
@property(readwrite, nonatomic) BOOL shouldRepeat;

/** This specifies the progress of the process on a scale from 0 to 1.0. A value of 0 means the process has not yet begun, A value of 1.0 means the conversaion is complete.
    This property is not key-value observable.
 */
@property(readonly, nonatomic) float progress;

/** This is used to send the delete Movie did complete playing alert
 */
@property (readwrite, nonatomic, assign) id <GPUImageMovieDelegate>delegate;

@property (readonly, nonatomic) AVAssetReader *assetReader;
@property (readonly, nonatomic) BOOL audioEncodingIsFinished;
@property (readonly, nonatomic) BOOL videoEncodingIsFinished;

• 作用：GPUImageMovie类继承了GPUImageOutput类，一般作为响应链的源头，可以通过url、playerItem、asset初始化。

7. GPUImagePicture

• 继承与属性

@interface GPUImagePicture : GPUImageOutput
{
    CGSize pixelSizeOfImage;
    BOOL hasProcessedImage;
    
    dispatch_semaphore_t imageUpdateSemaphore;
}

• 作用：GPUImagePicture是PGUImage的图像处理类，继承GPUImageOutput，一般作为响应链的源头。

几个属性

• pixelSizeOfImage 图像的像素大小。
• hasProcessedImage 图像是否已处理。
• imageUpdateSemaphore图像处理的GCD信号量。

几个方法

• - (id)initWithCGImage:smoothlyScaleOutput:用源图像newImageSource和是否采用mipmaps来初始化GPUImagePicture。
  如果图像大小超过OpenGL ES最大纹理宽高，或者使用mipmaps，或者图像数据是浮点型、颜色空间不对等都会采用CoreGraphics重新绘制图像。
  然后通过glTexImage2D把图像数据发送给GPU，最后释放掉CPU的图像数据。
• - (BOOL)processImageWithCompletionHandler:;通知targets处理图像，并在完成后调用complete代码块。在处理开始时，会标记hasProcessedImage为YES，并调用dispatch_semaphore_wait()，确定上次处理已经完成，否则取消这次处理。
• - (void)addTarget: atTextureLocation:;添加target到响应链。如果hasProcessedImage为YES，表示图像已经处理完毕，直接设置targets的InputSize，并调用newFrameReadyAtTime()通知target。

8. GPUImageRawDataInput

• 继承与属性

@interface GPUImageRawDataInput : GPUImageOutput
{
    CGSize uploadedImageSize;
    
    dispatch_semaphore_t dataUpdateSemaphore;
}

• 作用：GPUImageRawDataInput类继承GPUImageOutput类，可以接受二进制数据，按照特定的颜色格式，把数据转成图像并传入响应链；GPUImageRawDataInput不会对传入的数据copied或者retained，但你不需要在使用完之后去释放；二进制数据发送到GPU的纹理单元，默认的颜色格式是BGRA和整型数据。

  • 上传图片的逻辑：先申请outputframebuffer ，然后绑定纹理，最后用glTexImage2D 上传图像数据到GPU。
  • processData方法：处理图片；如果上一次操作还未完成，则直接返回。

9. GPUImageRawDataOutput

• 继承与属性

#if TARGET_IPHONE_SIMULATOR || TARGET_OS_IPHONE
@interface GPUImageRawDataOutput : NSObject <GPUImageInput> {
    CGSize imageSize;
    GPUImageRotationMode inputRotation;
    BOOL outputBGRA;
}
#else
@interface GPUImageRawDataOutput : NSObject <GPUImageInput> {
    CGSize imageSize;
    GPUImageRotationMode inputRotation;
    BOOL outputBGRA;
}
#endif

@property(readonly) GLubyte *rawBytesForImage;
@property(nonatomic, copy) void(^newFrameAvailableBlock)(void);
@property(nonatomic) BOOL enabled;

• 作用：GPUImageRawDataOutput类实现协议GPUImageInput，可以接受响应链的图像信息，并且以二进制的格式返回数据；

  • rawBytesForImage属性： 二进制数据的指针；
  • GPUByteColorVector结构体：RGBA颜色空间结构体，便于读取二进制数据；
  • supportsFastTextureUpload用的BGRA的颜色格式；
    如果需要输出RGBA，则可以对BGRA格式再做一次RGBA->BRGA的颜色转换；RGBA -> BGRA 的操作如下：
    texture2D(inputImageTexture, textureCoordinate).bgra;
  • lockNextFramebuffer属性：标志是否要读取图像信息如果为YES，会调用CVPixelBufferLockBaseAddress锁住对应的CVPixelBufferRef；

10. GPUImageMovieWriter

• 继承与属性

@interface GPUImageMovieWriter : NSObject <GPUImageInput>
{
    BOOL alreadyFinishedRecording;
    
    NSURL *movieURL;
    NSString *fileType;
    AVAssetWriter *assetWriter;
    AVAssetWriterInput *assetWriterAudioInput;
    AVAssetWriterInput *assetWriterVideoInput;
    AVAssetWriterInputPixelBufferAdaptor *assetWriterPixelBufferInput;
    
    GPUImageContext *_movieWriterContext;
    CVPixelBufferRef renderTarget;
    CVOpenGLESTextureRef renderTexture;

    CGSize videoSize;
    GPUImageRotationMode inputRotation;
}

@property(readwrite, nonatomic) BOOL hasAudioTrack;
@property(readwrite, nonatomic) BOOL shouldPassthroughAudio;
@property(readwrite, nonatomic) BOOL shouldInvalidateAudioSampleWhenDone;
@property(nonatomic, copy) void(^completionBlock)(void);
@property(nonatomic, copy) void(^failureBlock)(NSError*);
@property(nonatomic, assign) id<GPUImageMovieWriterDelegate> delegate;
@property(readwrite, nonatomic) BOOL encodingLiveVideo;
@property(nonatomic, copy) BOOL(^videoInputReadyCallback)(void);
@property(nonatomic, copy) BOOL(^audioInputReadyCallback)(void);
@property(nonatomic, copy) void(^audioProcessingCallback)(SInt16 **samplesRef, CMItemCount numSamplesInBuffer);
@property(nonatomic) BOOL enabled;
@property(nonatomic, readonly) AVAssetWriter *assetWriter;
@property(nonatomic, readonly) CMTime duration;
@property(nonatomic, assign) CGAffineTransform transform;
@property(nonatomic, copy) NSArray *metaData;
@property(nonatomic, assign, getter = isPaused) BOOL paused;
@property(nonatomic, retain) GPUImageContext *movieWriterContext;

• 作用：GPUImageMovieWriter类实现GPUImageInput协议，一般作为响应链的终点。shouldPassthroughAudio表示是否使用源音源。
  movieFile.audioEncodingTarget = movieWriter;表示音频来源是文件

11. GPUImageFilterPipeline

• 继承与属性

@interface GPUImageFilterPipeline : NSObject
{
    NSString *stringValue;
}

@property (strong) NSMutableArray *filters;

@property (strong) GPUImageOutput *input;
@property (strong) id <GPUImageInput> output;

• 作用：GPUImageFilterPipeline类是滤镜通道，把inputs的滤镜组合起来，然后添加output为最后的输出目标。

  • filters为输入的滤镜，output为输出目标；
  • 把filters的滤镜按照链表的形式串联起来。

12. GPUImageTextureOutput

• 继承与属性

@interface GPUImageTextureOutput : NSObject <GPUImageInput>
{
    GPUImageFramebuffer *firstInputFramebuffer;
}

@property(readwrite, unsafe_unretained, nonatomic) id<GPUImageTextureOutputDelegate> delegate;
@property(readonly) GLuint texture;
@property(nonatomic) BOOL enabled;

• 作用：GPUImageTextureOutput类实现GPUImageInput协议，可以接受响应链的图像，并返回对应的OpenGL ES纹理。

• delegate属性：实现了GPUImageTextureOutputDelegate协议的回调对象；

• texture属性：OpenGL ES的纹理，只读；

• enabled属性：是否有效，默认为有效；

• doneWithTexture方法：结束处理纹理图像，解锁firstInputFramebuffer。

13. GPUImageFilterGroup

• 继承与属性

@interface GPUImageFilterGroup : GPUImageOutput <GPUImageInput>
{
    NSMutableArray *filters;
    BOOL isEndProcessing;
}

@property(readwrite, nonatomic, strong) GPUImageOutput<GPUImageInput> *terminalFilter;
@property(readwrite, nonatomic, strong) NSArray *initialFilters;
@property(readwrite, nonatomic, strong) GPUImageOutput<GPUImageInput> *inputFilterToIgnoreForUpdates; 

• 作用：组合滤镜，添加滤镜的顺序不同，效果也不同。

14. GPUImageTextureInput

• 继承与属性

@interface GPUImageTextureInput : GPUImageOutput
{
    CGSize textureSize;
}

// Initialization and teardown
- (id)initWithTexture:(GLuint)newInputTexture size:(CGSize)newTextureSize;

// Image rendering
- (void)processTextureWithFrameTime:(CMTime)frameTime;

@end

• 作用：GPUImageTextureInput类继承GPUImageOutput类，可以作为响应链的起点，把OpenGL ES纹理对应的纹理信息导入响应链处理。textureSize属性为纹理尺寸；
  初始化的时候，分配一个GPUImageFramebuffer，缓存纹理单元的信息；process的时候直接调用targets对应的就绪方法，因为图像信息就在OpenGL ES控制内存中。

GPUImageTextureOutput和 GPUImageTextureInput用于 向OpenGL ES 输入或者输出纹理，把GPUImage的输出作为OpenGL ES的纹理或者把OpenGL ES的输出作为GPUImage的纹理输入。

15. GPUImageUIElement

• 继承与属性

@interface GPUImageUIElement : GPUImageOutput

// Initialization and teardown
- (id)initWithView:(UIView *)inputView;
- (id)initWithLayer:(CALayer *)inputLayer;

// Layer management
- (CGSize)layerSizeInPixels;
- (void)update;
- (void)updateUsingCurrentTime;
- (void)updateWithTimestamp:(CMTime)frameTime;

@end

• 作用：GPUImageUIElement继承GPUImageOutput类，作为响应链的源头。通过CoreGraphics把UIView渲染到图像，并通过glTexImage2D绑定到outputFramebuffer指定的纹理，最后通知targets纹理就绪。

16. GPUImageBuffer

• 继承与属性

@interface GPUImageBuffer : GPUImageFilter
{
    NSMutableArray *bufferedFramebuffers;
}

@property(readwrite, nonatomic) NSUInteger bufferSize;

@end

17. GPUImageFramebuffer

• 继承与属性

@interface GPUImageFramebuffer : NSObject

@property(readonly) CGSize size;
@property(readonly) GPUTextureOptions textureOptions;
@property(readonly) GLuint texture;
@property(readonly) BOOL missingFramebuffer;

• 作用：假设我们自定义一个OpenGL ES程序来处理图片，那么会有以下几个步骤：
  • 初始化OpenGL ES环境，编译、链接顶点着色器和片元着色器；
  • 缓存顶点、纹理坐标数据，传送图像数据到GPU；
  • 绘制图元到特定的帧缓存；
  • 在帧缓存取出绘制的图像。
    GPUImageFilter负责的是第一、二、三步。
    GPUImageFramebuffer负责是第四步。

18. GPUImageFramebufferCache

• 继承与属性

@interface GPUImageFramebufferCache : NSObject

// Framebuffer management
- (GPUImageFramebuffer *)fetchFramebufferForSize:(CGSize)framebufferSize textureOptions:(GPUTextureOptions)textureOptions onlyTexture:(BOOL)onlyTexture;
- (GPUImageFramebuffer *)fetchFramebufferForSize:(CGSize)framebufferSize onlyTexture:(BOOL)onlyTexture;
- (void)returnFramebufferToCache:(GPUImageFramebuffer *)framebuffer;
- (void)purgeAllUnassignedFramebuffers;
- (void)addFramebufferToActiveImageCaptureList:(GPUImageFramebuffer *)framebuffer;
- (void)removeFramebufferFromActiveImageCaptureList:(GPUImageFramebuffer *)framebuffer;

@end

• 作用：GPUImageFramebufferCache是GPUImageFrameBuffer的管理类。

几个属性

• CacheframebufferCache缓存字典
• framebufferTypeCounts 缓存数量字典
• activeImageCaptureList 正在读取Image数据的
• GPUImageFrameBuffer列表
• framebufferCacheQueue 缓存队列

几个方法

• - (NSString *)hashForSize: textureOptions:onlyTexture:;
  根据size、textureOptions和onlyTexture，创建缓存字符串。
  缓存字符串+当前缓存数量形成framebufferCache缓存的key。
  如果找不到framebufferCache对应的数量，会创建新的缓存。

• - (void)returnFramebufferToCache:;回收缓存。根据size、textureOptions和onlyTexture，创建缓存字符串，缓存字符串+当前缓存数量形成framebufferCache缓存的key。（之所以会加上数量，是因为缓存字符串不唯一）

• - (void)addFramebufferToActiveImageCaptureList:;
-(void)removeFramebufferFromActiveImageCaptureList:
  这两个方法主要用于，当newCGImageFromFramebufferContents()读取帧缓存图像数据时，保持GPUImageFramebuffer的引用。并且读取完数据后，在dataProviderUnlockCallback()方法释放。

参考文章

1. GPUImage详细解析（九）图像的输入输出和滤镜通道
2. GPUImageStillCamera 摄像头-照相
3. iOS GPUImage研究总结

后记

未完，待续~~~