Skip to content

CachedRDDBuilder

CachedRDDBuilder is <> exclusively when InMemoryRelation leaf logical operator is created.

[[cachedColumnBuffers]] CachedRDDBuilder uses a RDD of <> that is either <<_cachedColumnBuffers, given>> or <>.

[[CachedBatch]] CachedRDDBuilder uses CachedBatch data structure with the following attributes:

  • [[numRows]] Number of rows
  • [[buffers]] Buffers (Array[Array[Byte]])
  • [[stats]] Statistics (InternalRow)

[[isCachedColumnBuffersLoaded]] CachedRDDBuilder uses isCachedColumnBuffersLoaded flag that is enabled (true) when the <<_cachedColumnBuffers, _cachedColumnBuffers>> is defined (not null). isCachedColumnBuffersLoaded is used exclusively when CacheManager is requested to recacheByCondition.

[[sizeInBytesStats]] CachedRDDBuilder uses sizeInBytesStats metric (LongAccumulator) to <> and when InMemoryRelation is requested to computeStats.

=== [[creating-instance]] Creating CachedRDDBuilder Instance

CachedRDDBuilder takes the following to be created:

  • [[useCompression]] useCompression flag
  • [[batchSize]] Batch size
  • [[storageLevel]] StorageLevel
  • [[cachedPlan]] Physical operator
  • [[tableName]] Table name
  • [[_cachedColumnBuffers]] RDD[CachedBatch] (default: null)

CachedRDDBuilder initializes the <>.

=== [[buildBuffers]] buildBuffers Internal Method

[source, scala]

buildBuffers(): RDD[CachedBatch]

buildBuffers...FIXME

NOTE: buildBuffers is used exclusively when CachedRDDBuilder is requested to <>.

clearCache

clearCache(
  blocking: Boolean = true): Unit

clearCache...FIXME

clearCache is used exclusively when CacheManager is requested to clearCache, uncacheQuery, and recacheByCondition.


Last update: 2020-10-21
Back to top