Search in sources :

Example 1 with TableAlreadyExistsException

use of org.apache.spark.sql.catalyst.analysis.TableAlreadyExistsException in project iceberg by apache.

the class SparkCatalog method stageCreate.

@Override
public StagedTable stageCreate(Identifier ident, StructType schema, Transform[] transforms, Map<String, String> properties) throws TableAlreadyExistsException {
    Schema icebergSchema = SparkSchemaUtil.convert(schema, useTimestampsWithoutZone);
    try {
        Catalog.TableBuilder builder = newBuilder(ident, icebergSchema);
        Transaction transaction = builder.withPartitionSpec(Spark3Util.toPartitionSpec(icebergSchema, transforms)).withLocation(properties.get("location")).withProperties(Spark3Util.rebuildCreateProperties(properties)).createTransaction();
        return new StagedSparkTable(transaction);
    } catch (AlreadyExistsException e) {
        throw new TableAlreadyExistsException(ident);
    }
}
Also used : TableAlreadyExistsException(org.apache.spark.sql.catalyst.analysis.TableAlreadyExistsException) Transaction(org.apache.iceberg.Transaction) AlreadyExistsException(org.apache.iceberg.exceptions.AlreadyExistsException) NamespaceAlreadyExistsException(org.apache.spark.sql.catalyst.analysis.NamespaceAlreadyExistsException) TableAlreadyExistsException(org.apache.spark.sql.catalyst.analysis.TableAlreadyExistsException) Schema(org.apache.iceberg.Schema) StagedSparkTable(org.apache.iceberg.spark.source.StagedSparkTable) TableCatalog(org.apache.spark.sql.connector.catalog.TableCatalog) CachingCatalog(org.apache.iceberg.CachingCatalog) HadoopCatalog(org.apache.iceberg.hadoop.HadoopCatalog) Catalog(org.apache.iceberg.catalog.Catalog)

Example 2 with TableAlreadyExistsException

use of org.apache.spark.sql.catalyst.analysis.TableAlreadyExistsException in project iceberg by apache.

the class SparkCatalog method createTable.

@Override
public SparkTable createTable(Identifier ident, StructType schema, Transform[] transforms, Map<String, String> properties) throws TableAlreadyExistsException {
    Schema icebergSchema = SparkSchemaUtil.convert(schema, useTimestampsWithoutZone);
    try {
        Catalog.TableBuilder builder = newBuilder(ident, icebergSchema);
        Table icebergTable = builder.withPartitionSpec(Spark3Util.toPartitionSpec(icebergSchema, transforms)).withLocation(properties.get("location")).withProperties(Spark3Util.rebuildCreateProperties(properties)).create();
        return new SparkTable(icebergTable, !cacheEnabled);
    } catch (AlreadyExistsException e) {
        throw new TableAlreadyExistsException(ident);
    }
}
Also used : TableAlreadyExistsException(org.apache.spark.sql.catalyst.analysis.TableAlreadyExistsException) StagedSparkTable(org.apache.iceberg.spark.source.StagedSparkTable) StagedTable(org.apache.spark.sql.connector.catalog.StagedTable) Table(org.apache.iceberg.Table) SparkTable(org.apache.iceberg.spark.source.SparkTable) AlreadyExistsException(org.apache.iceberg.exceptions.AlreadyExistsException) NamespaceAlreadyExistsException(org.apache.spark.sql.catalyst.analysis.NamespaceAlreadyExistsException) TableAlreadyExistsException(org.apache.spark.sql.catalyst.analysis.TableAlreadyExistsException) Schema(org.apache.iceberg.Schema) StagedSparkTable(org.apache.iceberg.spark.source.StagedSparkTable) SparkTable(org.apache.iceberg.spark.source.SparkTable) TableCatalog(org.apache.spark.sql.connector.catalog.TableCatalog) CachingCatalog(org.apache.iceberg.CachingCatalog) HadoopCatalog(org.apache.iceberg.hadoop.HadoopCatalog) Catalog(org.apache.iceberg.catalog.Catalog)

Example 3 with TableAlreadyExistsException

use of org.apache.spark.sql.catalyst.analysis.TableAlreadyExistsException in project iceberg by apache.

the class SparkCatalog method renameTable.

@Override
public void renameTable(Identifier from, Identifier to) throws NoSuchTableException, TableAlreadyExistsException {
    try {
        checkNotPathIdentifier(from, "renameTable");
        checkNotPathIdentifier(to, "renameTable");
        icebergCatalog.renameTable(buildIdentifier(from), buildIdentifier(to));
    } catch (org.apache.iceberg.exceptions.NoSuchTableException e) {
        throw new NoSuchTableException(from);
    } catch (AlreadyExistsException e) {
        throw new TableAlreadyExistsException(to);
    }
}
Also used : TableAlreadyExistsException(org.apache.spark.sql.catalyst.analysis.TableAlreadyExistsException) AlreadyExistsException(org.apache.iceberg.exceptions.AlreadyExistsException) NamespaceAlreadyExistsException(org.apache.spark.sql.catalyst.analysis.NamespaceAlreadyExistsException) TableAlreadyExistsException(org.apache.spark.sql.catalyst.analysis.TableAlreadyExistsException) NoSuchTableException(org.apache.spark.sql.catalyst.analysis.NoSuchTableException)

Example 4 with TableAlreadyExistsException

use of org.apache.spark.sql.catalyst.analysis.TableAlreadyExistsException in project iceberg by apache.

the class SparkSessionCatalog method stageCreateOrReplace.

@Override
public StagedTable stageCreateOrReplace(Identifier ident, StructType schema, Transform[] partitions, Map<String, String> properties) throws NoSuchNamespaceException {
    String provider = properties.get("provider");
    TableCatalog catalog;
    if (useIceberg(provider)) {
        if (asStagingCatalog != null) {
            return asStagingCatalog.stageCreateOrReplace(ident, schema, partitions, properties);
        }
        catalog = icebergCatalog;
    } else {
        catalog = getSessionCatalog();
    }
    // drop the table if it exists
    catalog.dropTable(ident);
    try {
        // create the table with the session catalog, then wrap it in a staged table that will delete to roll back
        Table sessionCatalogTable = catalog.createTable(ident, schema, partitions, properties);
        return new RollbackStagedTable(catalog, ident, sessionCatalogTable);
    } catch (TableAlreadyExistsException e) {
        // the table was deleted, but now already exists again. retry the replace.
        return stageCreateOrReplace(ident, schema, partitions, properties);
    }
}
Also used : TableAlreadyExistsException(org.apache.spark.sql.catalyst.analysis.TableAlreadyExistsException) StagedTable(org.apache.spark.sql.connector.catalog.StagedTable) Table(org.apache.spark.sql.connector.catalog.Table) TableCatalog(org.apache.spark.sql.connector.catalog.TableCatalog) StagingTableCatalog(org.apache.spark.sql.connector.catalog.StagingTableCatalog)

Example 5 with TableAlreadyExistsException

use of org.apache.spark.sql.catalyst.analysis.TableAlreadyExistsException in project iceberg by apache.

the class SparkSessionCatalog method stageReplace.

@Override
public StagedTable stageReplace(Identifier ident, StructType schema, Transform[] partitions, Map<String, String> properties) throws NoSuchNamespaceException, NoSuchTableException {
    String provider = properties.get("provider");
    TableCatalog catalog;
    if (useIceberg(provider)) {
        if (asStagingCatalog != null) {
            return asStagingCatalog.stageReplace(ident, schema, partitions, properties);
        }
        catalog = icebergCatalog;
    } else {
        catalog = getSessionCatalog();
    }
    // attempt to drop the table and fail if it doesn't exist
    if (!catalog.dropTable(ident)) {
        throw new NoSuchTableException(ident);
    }
    try {
        // create the table with the session catalog, then wrap it in a staged table that will delete to roll back
        Table table = catalog.createTable(ident, schema, partitions, properties);
        return new RollbackStagedTable(catalog, ident, table);
    } catch (TableAlreadyExistsException e) {
        // the table was deleted, but now already exists again. retry the replace.
        return stageReplace(ident, schema, partitions, properties);
    }
}
Also used : TableAlreadyExistsException(org.apache.spark.sql.catalyst.analysis.TableAlreadyExistsException) StagedTable(org.apache.spark.sql.connector.catalog.StagedTable) Table(org.apache.spark.sql.connector.catalog.Table) TableCatalog(org.apache.spark.sql.connector.catalog.TableCatalog) StagingTableCatalog(org.apache.spark.sql.connector.catalog.StagingTableCatalog) NoSuchTableException(org.apache.spark.sql.catalyst.analysis.NoSuchTableException)

Aggregations

TableAlreadyExistsException (org.apache.spark.sql.catalyst.analysis.TableAlreadyExistsException)5 TableCatalog (org.apache.spark.sql.connector.catalog.TableCatalog)4 AlreadyExistsException (org.apache.iceberg.exceptions.AlreadyExistsException)3 NamespaceAlreadyExistsException (org.apache.spark.sql.catalyst.analysis.NamespaceAlreadyExistsException)3 StagedTable (org.apache.spark.sql.connector.catalog.StagedTable)3 CachingCatalog (org.apache.iceberg.CachingCatalog)2 Schema (org.apache.iceberg.Schema)2 Catalog (org.apache.iceberg.catalog.Catalog)2 HadoopCatalog (org.apache.iceberg.hadoop.HadoopCatalog)2 StagedSparkTable (org.apache.iceberg.spark.source.StagedSparkTable)2 NoSuchTableException (org.apache.spark.sql.catalyst.analysis.NoSuchTableException)2 StagingTableCatalog (org.apache.spark.sql.connector.catalog.StagingTableCatalog)2 Table (org.apache.spark.sql.connector.catalog.Table)2 Table (org.apache.iceberg.Table)1 Transaction (org.apache.iceberg.Transaction)1 SparkTable (org.apache.iceberg.spark.source.SparkTable)1