Olive
 All Classes Namespaces Files Functions Variables Typedefs Enumerations Enumerator Macros
Public Member Functions | Private Member Functions | Private Attributes | List of all members
Cacher Class Reference

The Cacher class. More...

#include <cacher.h>

Inheritance diagram for Cacher:

Public Member Functions

 Cacher (Clip *c)
 Cacher Constructor. More...
 
void run ()
 The main QThread loop. More...
 
void Open ()
 Open the cacher. More...
 
void Cache (long playhead, bool scrubbing, QVector< Clip * > &nests, int playback_speed)
 Request a frame to be cached. More...
 
AVFrame * Retrieve ()
 Retrieve frame requested by Cache() More...
 
void Close (bool wait_for_finish)
 Close the cacher and free any allocated memory. More...
 
void ResetAudio ()
 Interrupt and reset audio state. More...
 
int media_width ()
 Retrieve current media width. More...
 
int media_height ()
 Retrieve current media height. More...
 
AVRational media_time_base ()
 Retrieve media time base. More...
 
ClipQueuequeue ()
 Get cacher queue object. More...
 

Private Member Functions

void OpenWorker ()
 Internal function for opening the file handles and decoder. More...
 
void CacheWorker ()
 Internal function for starting a cache cycle. More...
 
void CloseWorker ()
 Internal function for closing cacher. More...
 
void Reset ()
 Internal function for resetting audio state. More...
 
void SetRetrievedFrame (AVFrame *f)
 Internal function for setting retrieved_frame and waking up any threads waiting for it. More...
 
void WakeMainThread ()
 Internal function to wake an external calling thread. More...
 
int RetrieveFrameFromDecoder (AVFrame *f)
 Retrieve frame from decoder. More...
 
int RetrieveFrameAndProcess (AVFrame **f)
 Retrieve frame from decoder and run it through filter stack. More...
 
void CacheVideoWorker ()
 Internal video caching function. More...
 
void CacheAudioWorker ()
 Internal audio caching function. More...
 
bool IsReversed ()
 Internal function using the Cacher's known information to determine whether this media is playing in reverse. More...
 

Private Attributes

Clipclip
 Reference to the parent clip. Set in the constructor and never changed during this object's lifetime. More...
 
ClipQueue queue_
 Frame queue. More...
 
QWaitCondition wait_cond_
 Main wait condition. More...
 
QWaitCondition main_thread_wait_
 Main thread wait condition. More...
 
QMutex main_thread_lock_
 Main thread mutex. More...
 
QWaitCondition retrieve_wait_
 Retrieve() wait condition. More...
 
QMutex retrieve_lock_
 Retrieve() mutex. More...
 
bool reached_end
 Set and used by CacheAudioWorker if the decoder receives an EOF. More...
 
long playhead_
 Current Sequence playhead set by Cache() More...
 
bool scrubbing_
 Current Sequence scrubbing state set by Cache() More...
 
int playback_speed_
 Current Sequence playback speed set by Cache() More...
 
QVector< Clip * > nests_
 Current nested Sequence hierarchy set by Cache() More...
 
bool queued_
 Signal cache to continue operation after one cycle rather than wait for another signal. More...
 
bool interrupt_
 Interrupt the current cache cycle. More...
 
AVFormatContext * formatCtx
 FFmpeg format/file context - used for media decoding. More...
 
AVCodecContext * codecCtx
 FFmpeg decoder context - used for media decoding. More...
 
AVStream * stream
 FFmpeg stream - used for media decoding. More...
 
AVPacket * pkt
 FFmpeg packet - used for media decoding. More...
 
AVFrame * frame_
 FFmpeg frame - used for media decoding. More...
 
AVFrame * retrieved_frame = nullptr
 Retrieved frame reference for Retrieve() More...
 
AVFilterGraph * filter_graph
 FFmpeg filter stack. More...
 
AVFilterContext * buffersrc_ctx
 FFmpeg buffer source. More...
 
AVFilterContext * buffersink_ctx
 FFmpeg buffer sink. More...
 
AVCodec * codec
 FFmpeg codec reference. More...
 
AVDictionary * opts
 Options set by the cacher for FFmpeg's decoders (settings like multithreading or other optimizations) More...
 
bool audio_reset_
 Internal audio reset variable. More...
 
int64_t reverse_target_
 Internal reverse target variable. More...
 
int frame_sample_index_
 Internal frame sample index variable. More...
 
qint64 audio_buffer_write
 Internal audio buffer write variable. More...
 
long audio_target_frame
 Internal variable that holds the playhead the last time the audio state was reset. More...
 
bool caching_
 Main while loop condition to determine whether thread should continue looping. More...
 
bool is_valid_state_
 Internal variable for whether the current Cacher state is valid or not. More...
 

Detailed Description

The Cacher class.

For footage clips - usually the majority of clips - decoding can be strenuous on CPU and inconsistent in timing. As a result, we keep a memory cache of upcoming frames that we fill in a background thread so they can be retrieved from a rendering thread later. This class is the background thread filling up a clip's frame cache (also called a "queue" since video files are usually stored with frames in linear chronological order). It involves decoding routines to retrieve raw frames from the file (using libavformat/libavcodec), conversion routines to conform the raw frames to RGBA/S16LE for the rest of the workflow (using libavfilter/libswscale/libswresample), and memory handling routines for keeping the cache within limits defined by the user (see Config::upcoming_queue_type).

Generally the Cacher workflow starts by calling Open() which will start the thread, open a file handle, and create a decoding instance. Open() is usually called directly from the parent Clip's Clip::Open() and thus expects the Clip::state_change_lock to be locked. It will unlock it when it's finished opening and is ready to start caching, meaning Clip::state_change_lock can be used to synchronize threads.


For video:

After the Cacher has finished opening, request a frame by calling Cache(). Cache() will tell the Cacher information about the current playback state, most importantly the current place in time according to the Sequence's playhead. Cache() determines whether the requested frame is already in the queue, and then signals the Cacher thread to cache ahead if there's room in the queue (and also remove old frames that are no longer necessary). To retrieve the requested frame, call Retrieve().

If Cache() found the frame already in the queue, Retrieve() will return immediately with this frame. Otherwise Retrieve() may block while the cacher retrieves it. Therefore it is recommended never to call Retrieve() from the main thread. Retrieve() may also return nullptr if there was an issue, e.g. the cacher failed to retrieve the frame.

For audio:

After the Cacher has finished opening, calling Cache() will handle most of the work. It will decode the audio, convert to the correct sample rate and format, reverse or adjust speed if necessary, and send it to the audio buffer ready to be played by the output device. It is important to continually call Cache() as it doesn't get signalled when more samples are available in the audio buffer. Instead, it'll check every time it's called and fill as much of the buffer as it can.

If the user seeks, ResetAudio() must be called to signal the Cacher to interrupt the current audio stream and move somewhere else before continuing.


Finally, when the Cacher/parent Clip are no longer in use, call Close() to free all memory and file handling allocated for the cacher. You can choose whether to wait for Close() and all of its child processes to complete - e.g. if you need to change something with the Clip or attached Footage that changes how it opens and want to be thread-safe - or let the Close thread finish up on its own.

Cacher expects to be multithreaded and all of its public functions are thread-safe.

Constructor & Destructor Documentation

Cacher::Cacher ( Clip c)

Cacher Constructor.

Create Cacher object. The thread is not started here. To start it, call Open().

Parameters
c

Member Function Documentation

void Cacher::Cache ( long  playhead,
bool  scrubbing,
QVector< Clip * > &  nests,
int  playback_speed 
)

Request a frame to be cached.

For video, this function is part 1 of the Cache()/Retrieve() workflow. It signals the thread to start caching and provides a few other details about the playback state. For optimization it'll also check the frame queue if it already contains the requested frame and use it if so, potentially speeding up Retrieve() later on. Otherwise it'll interrupt any currently caching operation and signal it to start again. While Retrieve() will block until the correct frame is retrieved, this function will return fairly quickly (either immediately if the frame was found in the queue, or once the cacher has restarted caching if not). This means Cache() can be called from another thread and then that other thread can do other work while the cacher is retrieving the frame, finally calling Retrieve() once the frame is absolutely necessary.

For audio, this function will do all the work of signalling the thread to start caching and sending samples to the output audio buffer. It's used in tandem with ResetAudio() when the Timeline header is changed abruptly.

Parameters
playheadThe current Timeline played position in frames
scrubbingTRUE if the user is currently scrubbing. FALSE if not.
nestsA hierarchy of nested sequences, if the playback traversed any to get to this clip.
playback_speedThe current playback speed (controlled by Shuttle Left/Stop/Right)
void Cacher::CacheAudioWorker ( )
private

Internal audio caching function.

Perform one audio cache cycle. Retrieves audio from decoder, reverses and changes speed if necessary, and sends audio to the audio buffer which will later be sent to the audio output device.

void Cacher::CacheVideoWorker ( )
private

Internal video caching function.

Performs one video cache cycle. Seeks the media and cleans old frames from the queue if necessary. Decodes frames and adds them to the queue (after calculating whether they're necessary).

void Cacher::CacheWorker ( )
private

Internal function for starting a cache cycle.

This used to have more function, but now just differentiates between CacheVideoWorker() for video clips and CacheAudioWorker() for audio clips.

void Cacher::Close ( bool  wait_for_finish)

Close the cacher and free any allocated memory.

When the Cacher thread is no longer needed, Close() should be called in order to free system resources. This will signal the thread to exit gracefully, but will not delete the thread object since the cacher may need to be re-opened later by Open().

Parameters
wait_for_finishTRUE if this function should block the calling thread until the Clip has finished closing. Often necessary if the Clip is being closed specifically to make changes to it.
void Cacher::CloseWorker ( )
private

Internal function for closing cacher.

Called if the main thread loop in run() exits by setting caching_ to FALSE. Free's up handles and memory allocated by OpenWorker().

bool Cacher::IsReversed ( )
private

Internal function using the Cacher's known information to determine whether this media is playing in reverse.

int Cacher::media_height ( )

Retrieve current media height.

See media_width().

Only call after the thread has been opened by Open().

Returns

The true height of the current video file.

AVRational Cacher::media_time_base ( )

Retrieve media time base.

For some timing operations, it's necessary to use the source media's timebase. Similar to media_width() and media_height(), we need the accurate timebase from the file as a proxy's timebase may or may not be the same as the source file.

Only call after the thread has been opened by Open().

Returns

The timebase of the file.

int Cacher::media_width ( )

Retrieve current media width.

In some situations, the actual media we're using may be a different resolution to how we're treating it (e.g. lower resolution proxies). While most functions will happily treat the media as its original resolution, some processes will need the absolute resolution from the file which can be acquired here.

Only call after the thread has been opened by Open().

Returns

The true width of the current video file.

void Cacher::Open ( )

Open the cacher.

Starts the thread and all file/decode handlers. Really just sets some default values and starts the thread, which will in turn call OpenWorker() at the start of its functions.

Make sure Clip::state_change_lock is LOCKED before calling this function as the opening process will try to unlock it when it's finished (leading to a crash if it's not already locked).

void Cacher::OpenWorker ( )
private

Internal function for opening the file handles and decoder.

After the thread has started, it'll call this function to start all resources necessary for caching. Any FFmpeg decoding variables and filters are set up here.

This is fundamentally different from Open(), this is only meant to be called within the cacher thread and never from outside and doesn't start the thread like Open() does.

ClipQueue * Cacher::queue ( )

Get cacher queue object.

Returns

A pointer to the cacher's internal frame queue

void Cacher::Reset ( )
private

Internal function for resetting audio state.

This used to be a common function, but is now simply a legacy function for CacheAudioWorker(). Resets and flushes decoders and seeks to the correct timestamp.

void Cacher::ResetAudio ( )

Interrupt and reset audio state.

Used in tandem with Cache(), only for audio clips. Cache() will decode and send audio continually as it's repeatedly called. If the audio stream needs to be interrupted and moved somewhere else for any reason (e.g. the user seeked somewhere else), then it's necessary to call ResetAudio() to signal the cacher to seek to the next place indicated by Cache().

AVFrame * Cacher::Retrieve ( )

Retrieve frame requested by Cache()

Part 2 of the Cache()/Retrieve() workflow, only used for video. Whichever frame was requested by Cache(), this function will try to retrieve it. In most cases, this function will be pretty quick as the frame will be available immediately from Cache()'s optimization or the cacher thread will be close to retrieving the correct frame anyway. However it does block for however long it takes to retrieve the correct frame (if the cacher is running) so it's not recommended to call this from any main/GUI thread.

Returns

The frame requested by Cache(), or nullptr if there was an error (e.g. the cacher wasn't running and no frame was available).

int Cacher::RetrieveFrameAndProcess ( AVFrame **  f)
private

Retrieve frame from decoder and run it through filter stack.

Retrieves the next decoded frame and runs it through the AVFilter stack to create an RGBA frame compatible with the rest of the pipeline and OpenGL. Use this function if you need a ready-made frame.

Parameters
fA pointer to an AVFrame object. It does not need to be allocated, as this function allocates an AVFrame itself. You'll also need to free it later with av_frame_free() (though ClipQueue will do this automatically if the frame is added to it).
Returns

FFmpeg error code (>= 0 on success, a negative error code on failure)

int Cacher::RetrieveFrameFromDecoder ( AVFrame *  f)
private

Retrieve frame from decoder.

Retrieves the next decoded frame from the decoder. Depending on the source media, this frame may or may not be suitable for usage later in the pipeline as it may or may not be the correct pixel/sample format. For a suitable frame for the pipeline, use RetrieveFrameAndProcess() instead (which in turn uses this function anyway).

Parameters
fFrame buffer to decode frame into
Returns

FFmpeg error code (>= 0 on success, a negative error code on failure)

void Cacher::run ( )

The main QThread loop.

Once the thread has started, all Cacher functions will be called from here until the Cacher closes at which point it will close and exit gracefully.

void Cacher::SetRetrievedFrame ( AVFrame *  f)
private

Internal function for setting retrieved_frame and waking up any threads waiting for it.

Parameters
fThe frame to set as the retrieved frame.
void Cacher::WakeMainThread ( )
private

Internal function to wake an external calling thread.

In some situations, Cache() may wait for the cacher to respond before returning. This is to assist in thread synchronization, making sure the cacher has started working and has locked any resources it needs before any other threads can access them (e.g. with a function like Retrieve() ). This must be called at the start of any CacheVideoWorker() or CacheAudioWorker() control paths to ensure the render thread doesn't get stuck.

Member Data Documentation

qint64 Cacher::audio_buffer_write
private

Internal audio buffer write variable.

Used by CacheAudioWorker() to mark which part of the audio buffer to write to

bool Cacher::audio_reset_
private

Internal audio reset variable.

Set by AudioReset() and read by CacheAudioWorker() when the audio state needs to be interrupted and reset.

long Cacher::audio_target_frame
private

Internal variable that holds the playhead the last time the audio state was reset.

AVFilterContext* Cacher::buffersink_ctx
private

FFmpeg buffer sink.

Converted/filtered frames are retrieved from here and sent to Cacher::queue.

AVFilterContext* Cacher::buffersrc_ctx
private

FFmpeg buffer source.

Raw decoded frames are added to this for conversion/filtering

bool Cacher::caching_
private

Main while loop condition to determine whether thread should continue looping.

Open() sets this to TRUE, Close() sets this to FALSE. If it's false, the main loop in run() will exit and the thread will exit cleanly. It's not recommended to set this variable directly, use Open() and Close() instead.

Clip* Cacher::clip
private

Reference to the parent clip. Set in the constructor and never changed during this object's lifetime.

AVCodec* Cacher::codec
private

FFmpeg codec reference.

AVCodecContext* Cacher::codecCtx
private

FFmpeg decoder context - used for media decoding.

AVFilterGraph* Cacher::filter_graph
private

FFmpeg filter stack.

Used for conversion from the media's pixel format to RGBA for OpenGL. Also any other FFmpeg filters are implemented here if necessary (e.g. yadif for deinterlacing). GLSL effects are preferred when available since FFmpeg filters aren't always fast enough for realtime playback.

AVFormatContext* Cacher::formatCtx
private

FFmpeg format/file context - used for media decoding.

AVFrame* Cacher::frame_
private

FFmpeg frame - used for media decoding.

This is usually used as a raw decoded frame before the RGBA conversion/AVFilter stack. Converted/filtered frames go into Cacher::queue.

int Cacher::frame_sample_index_
private

Internal frame sample index variable.

Used by CacheAudioWorker() to mark which part of the audio frame to read from

bool Cacher::interrupt_
private

Interrupt the current cache cycle.

A cache cycle will cache several frames at a time. Since decoding can be strenuous and time consuming, the cycle can be interrupted if it needs to abruptly start caching somewhere else. Best used in tandem with queued_ to automatically start the next cache cycle.

bool Cacher::is_valid_state_
private

Internal variable for whether the current Cacher state is valid or not.

If there was an error opening the Cacher for any reason, this will be false.

QMutex Cacher::main_thread_lock_
private

Main thread mutex.

Used with main_thread_wait_ to block Cache() while waiting for a response from the cacher thread.

QWaitCondition Cacher::main_thread_wait_
private

Main thread wait condition.

Used with main_thread_lock_ to block Cache() while waiting for a response from the cacher thread.

QVector<Clip*> Cacher::nests_
private

Current nested Sequence hierarchy set by Cache()

AVDictionary* Cacher::opts
private

Options set by the cacher for FFmpeg's decoders (settings like multithreading or other optimizations)

AVPacket* Cacher::pkt
private

FFmpeg packet - used for media decoding.

int Cacher::playback_speed_
private

Current Sequence playback speed set by Cache()

long Cacher::playhead_
private

Current Sequence playhead set by Cache()

ClipQueue Cacher::queue_
private

Frame queue.

Valid fames are cached into this, which also does memory handling when necessary.

bool Cacher::queued_
private

Signal cache to continue operation after one cycle rather than wait for another signal.

Each cycle of the cacher thread (see run()) will set this to false in the beginning. Each call of Cache() will set this to TRUE. If this variable is TRUE, the cacher won't wait for another signal before starting the next cache cycle, and will instead just start it.

Used if Cache() is called and interrupts the cacher while it's already running so that the cacher will restart itself automatically rather than wait for the next cache signal.

bool Cacher::reached_end
private

Set and used by CacheAudioWorker if the decoder receives an EOF.

Deprecated. CacheAudioWorker() is functional but probably should be rewritten.

QMutex Cacher::retrieve_lock_
private

Retrieve() mutex.

Used with retrieve_wait_ to block Retrieve() if the cacher hasn't retrieved the correct frame yet.

QWaitCondition Cacher::retrieve_wait_
private

Retrieve() wait condition.

Used with retrieve_lock_ to block Retrieve() if the cacher hasn't retrieved the correct frame yet.

AVFrame* Cacher::retrieved_frame = nullptr
private

Retrieved frame reference for Retrieve()

If a frame was found by either Cache() or CacheVideoWorker(), it's set here. If no frame is ready yet, this is set to nullptr.

int64_t Cacher::reverse_target_
private

Internal reverse target variable.

Used by CacheAudioWorker() to stitch audio frames together when reversing. Stores the current frame's timestamp so it knows how much to decode up to when it backtracks and decodes the next samples.

bool Cacher::scrubbing_
private

Current Sequence scrubbing state set by Cache()

AVStream* Cacher::stream
private

FFmpeg stream - used for media decoding.

QWaitCondition Cacher::wait_cond_
private

Main wait condition.

Used with Clip::cache_lock as the main block while the the Cacher thread isn't running. Wake this condition to start caching.


The documentation for this class was generated from the following files: