Examples with DeltaTableV2 org.apache.spark.sql.delta.catalog.DeltaTableV2 used on opensource projects

Example 1 with DeltaTableV2

use of org.apache.spark.sql.delta.catalog.DeltaTableV2 in project OpenLineage by OpenLineage.

the class DeltaHandler method getDatasetVersion.

@SneakyThrows
public Optional<String> getDatasetVersion(TableCatalog tableCatalog, Identifier identifier, Map<String, String> properties) {
    DeltaCatalog deltaCatalog = (DeltaCatalog) tableCatalog;
    Table table = deltaCatalog.loadTable(identifier);
    if (table instanceof DeltaTableV2) {
        DeltaTableV2 deltaTable = (DeltaTableV2) table;
        return Optional.of(Long.toString(deltaTable.snapshot().version()));
    }
    return Optional.empty();
}

Example 2 with DeltaTableV2

use of org.apache.spark.sql.delta.catalog.DeltaTableV2 in project OpenLineage by OpenLineage.

the class DeltaHandlerTest method testGetVersionString.

@Test
public void testGetVersionString() {
    DeltaCatalog deltaCatalog = mock(DeltaCatalog.class);
    DeltaTableV2 deltaTable = mock(DeltaTableV2.class, RETURNS_DEEP_STUBS);
    Identifier identifier = Identifier.of(new String[] { "database", "schema" }, "table");
    DeltaHandler deltaHandler = new DeltaHandler();
    when(deltaCatalog.loadTable(identifier)).thenReturn(deltaTable);
    when(deltaTable.snapshot().version()).thenReturn(2L);
    Optional<String> version = deltaHandler.getDatasetVersion(deltaCatalog, identifier, Collections.emptyMap());
    assertTrue(version.isPresent());
    assertEquals(version.get(), "2");
}