Search in sources :

Example 6 with StagedSparkTable

use of org.apache.iceberg.spark.source.StagedSparkTable in project iceberg by apache.

the class SparkCatalog method stageReplace.

@Override
public StagedTable stageReplace(Identifier ident, StructType schema, Transform[] transforms, Map<String, String> properties) throws NoSuchTableException {
    Schema icebergSchema = SparkSchemaUtil.convert(schema, useTimestampsWithoutZone);
    try {
        Catalog.TableBuilder builder = newBuilder(ident, icebergSchema);
        Transaction transaction = builder.withPartitionSpec(Spark3Util.toPartitionSpec(icebergSchema, transforms)).withLocation(properties.get("location")).withProperties(Spark3Util.rebuildCreateProperties(properties)).replaceTransaction();
        return new StagedSparkTable(transaction);
    } catch (org.apache.iceberg.exceptions.NoSuchTableException e) {
        throw new NoSuchTableException(ident);
    }
}
Also used : Transaction(org.apache.iceberg.Transaction) Schema(org.apache.iceberg.Schema) StagedSparkTable(org.apache.iceberg.spark.source.StagedSparkTable) NoSuchTableException(org.apache.spark.sql.catalyst.analysis.NoSuchTableException) TableCatalog(org.apache.spark.sql.connector.catalog.TableCatalog) CachingCatalog(org.apache.iceberg.CachingCatalog) HadoopCatalog(org.apache.iceberg.hadoop.HadoopCatalog) Catalog(org.apache.iceberg.catalog.Catalog)

Aggregations

StagedSparkTable (org.apache.iceberg.spark.source.StagedSparkTable)6 CachingCatalog (org.apache.iceberg.CachingCatalog)3 Schema (org.apache.iceberg.Schema)3 Transaction (org.apache.iceberg.Transaction)3 Catalog (org.apache.iceberg.catalog.Catalog)3 AlreadyExistsException (org.apache.iceberg.exceptions.AlreadyExistsException)3 HadoopCatalog (org.apache.iceberg.hadoop.HadoopCatalog)3 TableCatalog (org.apache.spark.sql.connector.catalog.TableCatalog)3 Snapshot (org.apache.iceberg.Snapshot)2 Table (org.apache.iceberg.Table)2 TableIdentifier (org.apache.spark.sql.catalyst.TableIdentifier)2 BaseMigrateTableActionResult (org.apache.iceberg.actions.BaseMigrateTableActionResult)1 BaseSnapshotTableActionResult (org.apache.iceberg.actions.BaseSnapshotTableActionResult)1 MigrateTable (org.apache.iceberg.actions.MigrateTable)1 SnapshotTable (org.apache.iceberg.actions.SnapshotTable)1 NoSuchNamespaceException (org.apache.iceberg.exceptions.NoSuchNamespaceException)1 NoSuchTableException (org.apache.iceberg.exceptions.NoSuchTableException)1 NamespaceAlreadyExistsException (org.apache.spark.sql.catalyst.analysis.NamespaceAlreadyExistsException)1 NoSuchTableException (org.apache.spark.sql.catalyst.analysis.NoSuchTableException)1 TableAlreadyExistsException (org.apache.spark.sql.catalyst.analysis.TableAlreadyExistsException)1