CachedRDDBuilder¶
CachedRDDBuilder
is <
[[cachedColumnBuffers]] CachedRDDBuilder
uses a RDD
of <
[[CachedBatch]] CachedRDDBuilder
uses CachedBatch
data structure with the following attributes:
- [[numRows]] Number of rows
- [[buffers]] Buffers (
Array[Array[Byte]]
) - [[stats]] Statistics (InternalRow)
[[isCachedColumnBuffersLoaded]] CachedRDDBuilder
uses isCachedColumnBuffersLoaded
flag that is enabled (true
) when the <<_cachedColumnBuffers, _cachedColumnBuffers>> is defined (not null
). isCachedColumnBuffersLoaded
is used exclusively when CacheManager
is requested to recacheByCondition.
[[sizeInBytesStats]] CachedRDDBuilder
uses sizeInBytesStats
metric (LongAccumulator
) to <InMemoryRelation
is requested to computeStats.
=== [[creating-instance]] Creating CachedRDDBuilder Instance
CachedRDDBuilder
takes the following to be created:
- [[useCompression]]
useCompression
flag - [[batchSize]] Batch size
- [[storageLevel]]
StorageLevel
- [[cachedPlan]] Physical operator
- [[tableName]] Table name
- [[_cachedColumnBuffers]]
RDD[CachedBatch]
(default:null
)
CachedRDDBuilder
initializes the <
=== [[buildBuffers]] buildBuffers
Internal Method
[source, scala]¶
buildBuffers(): RDD[CachedBatch]¶
buildBuffers
...FIXME
NOTE: buildBuffers
is used exclusively when CachedRDDBuilder
is requested to <
clearCache¶
clearCache(
blocking: Boolean = true): Unit
clearCache
...FIXME
clearCache
is used exclusively when CacheManager
is requested to clearCache, uncacheQuery, and recacheByCondition.