RelationProvider — Relations with Pre-Defined Schema¶

RelationProvider is an abstraction of providers for relational data sources with schema inference (schema discovery). In other words, the schema of a relation is pre-defined (fixed), and a user-specified schema is not allowed.

Contract¶

createRelation¶

createRelation(
  sqlContext: SQLContext,
  parameters: Map[String, String]): BaseRelation

Creates a BaseRelation (given parameters and SQLContext)

Used when:

• DataSource is requested to resolve a relation

Implementations¶

• JdbcRelationProvider
• KafkaSourceProvider

CreatableRelationProvider¶

It is a common pattern while developing a custom data source to use RelationProvider.createRelation with CreatableRelationProvider when requested for a relation (after writing out a structured query).