Search in sources :

Example 11 with DatasetServiceSettings

use of com.google.cloud.aiplatform.v1.DatasetServiceSettings in project java-aiplatform by googleapis.

the class CreateDatasetVideoSample method createDatasetSample.

static void createDatasetSample(String datasetVideoDisplayName, String project) throws IOException, InterruptedException, ExecutionException, TimeoutException {
    DatasetServiceSettings datasetServiceSettings = DatasetServiceSettings.newBuilder().setEndpoint("us-central1-aiplatform.googleapis.com:443").build();
    // the "close" method on the client to safely clean up any remaining background resources.
    try (DatasetServiceClient datasetServiceClient = DatasetServiceClient.create(datasetServiceSettings)) {
        String location = "us-central1";
        String metadataSchemaUri = "gs://google-cloud-aiplatform/schema/dataset/metadata/video_1.0.0.yaml";
        LocationName locationName = LocationName.of(project, location);
        Dataset dataset = Dataset.newBuilder().setDisplayName(datasetVideoDisplayName).setMetadataSchemaUri(metadataSchemaUri).build();
        OperationFuture<Dataset, CreateDatasetOperationMetadata> datasetFuture = datasetServiceClient.createDatasetAsync(locationName, dataset);
        System.out.format("Operation name: %s\n", datasetFuture.getInitialFuture().get().getName());
        System.out.println("Waiting for operation to finish...");
        Dataset datasetResponse = datasetFuture.get(300, TimeUnit.SECONDS);
        System.out.println("Create Dataset Video Response");
        System.out.format("Name: %s\n", datasetResponse.getName());
        System.out.format("Display Name: %s\n", datasetResponse.getDisplayName());
        System.out.format("Metadata Schema Uri: %s\n", datasetResponse.getMetadataSchemaUri());
        System.out.format("Metadata: %s\n", datasetResponse.getMetadata());
        System.out.format("Create Time: %s\n", datasetResponse.getCreateTime());
        System.out.format("Update Time: %s\n", datasetResponse.getUpdateTime());
        System.out.format("Labels: %s\n", datasetResponse.getLabelsMap());
    }
}
Also used : DatasetServiceSettings(com.google.cloud.aiplatform.v1.DatasetServiceSettings) CreateDatasetOperationMetadata(com.google.cloud.aiplatform.v1.CreateDatasetOperationMetadata) Dataset(com.google.cloud.aiplatform.v1.Dataset) DatasetServiceClient(com.google.cloud.aiplatform.v1.DatasetServiceClient) LocationName(com.google.cloud.aiplatform.v1.LocationName)

Example 12 with DatasetServiceSettings

use of com.google.cloud.aiplatform.v1.DatasetServiceSettings in project java-aiplatform by googleapis.

the class CreateDatasetTabularBigquerySample method createDatasetTableBigquery.

static void createDatasetTableBigquery(String project, String bigqueryDisplayName, String bigqueryUri) throws IOException, ExecutionException, InterruptedException, TimeoutException {
    DatasetServiceSettings settings = DatasetServiceSettings.newBuilder().setEndpoint("us-central1-aiplatform.googleapis.com:443").build();
    // the "close" method on the client to safely clean up any remaining background resources.
    try (DatasetServiceClient datasetServiceClient = DatasetServiceClient.create(settings)) {
        String location = "us-central1";
        String metadataSchemaUri = "gs://google-cloud-aiplatform/schema/dataset/metadata/tables_1.0.0.yaml";
        LocationName locationName = LocationName.of(project, location);
        String jsonString = "{\"input_config\": {\"bigquery_source\": {\"uri\": \"" + bigqueryUri + "\"}}}";
        Value.Builder metaData = Value.newBuilder();
        JsonFormat.parser().merge(jsonString, metaData);
        Dataset dataset = Dataset.newBuilder().setDisplayName(bigqueryDisplayName).setMetadataSchemaUri(metadataSchemaUri).setMetadata(metaData).build();
        OperationFuture<Dataset, CreateDatasetOperationMetadata> datasetFuture = datasetServiceClient.createDatasetAsync(locationName, dataset);
        System.out.format("Operation name: %s\n", datasetFuture.getInitialFuture().get().getName());
        System.out.println("Waiting for operation to finish...");
        Dataset datasetResponse = datasetFuture.get(300, TimeUnit.SECONDS);
        System.out.println("Create Dataset Table Bigquery sample");
        System.out.format("Name: %s\n", datasetResponse.getName());
        System.out.format("Display Name: %s\n", datasetResponse.getDisplayName());
        System.out.format("Metadata Schema Uri: %s\n", datasetResponse.getMetadataSchemaUri());
        System.out.format("Metadata: %s\n", datasetResponse.getMetadata());
    }
}
Also used : DatasetServiceSettings(com.google.cloud.aiplatform.v1.DatasetServiceSettings) CreateDatasetOperationMetadata(com.google.cloud.aiplatform.v1.CreateDatasetOperationMetadata) Dataset(com.google.cloud.aiplatform.v1.Dataset) DatasetServiceClient(com.google.cloud.aiplatform.v1.DatasetServiceClient) Value(com.google.protobuf.Value) LocationName(com.google.cloud.aiplatform.v1.LocationName)

Example 13 with DatasetServiceSettings

use of com.google.cloud.aiplatform.v1.DatasetServiceSettings in project java-aiplatform by googleapis.

the class CreateDatasetTabularGcsSample method createDatasetTableGcs.

static void createDatasetTableGcs(String project, String datasetDisplayName, String gcsSourceUri) throws IOException, ExecutionException, InterruptedException, TimeoutException {
    DatasetServiceSettings settings = DatasetServiceSettings.newBuilder().setEndpoint("us-central1-aiplatform.googleapis.com:443").build();
    // the "close" method on the client to safely clean up any remaining background resources.
    try (DatasetServiceClient datasetServiceClient = DatasetServiceClient.create(settings)) {
        String location = "us-central1";
        String metadataSchemaUri = "gs://google-cloud-aiplatform/schema/dataset/metadata/tables_1.0.0.yaml";
        LocationName locationName = LocationName.of(project, location);
        String jsonString = "{\"input_config\": {\"gcs_source\": {\"uri\": [\"" + gcsSourceUri + "\"]}}}";
        Value.Builder metaData = Value.newBuilder();
        JsonFormat.parser().merge(jsonString, metaData);
        Dataset dataset = Dataset.newBuilder().setDisplayName(datasetDisplayName).setMetadataSchemaUri(metadataSchemaUri).setMetadata(metaData).build();
        OperationFuture<Dataset, CreateDatasetOperationMetadata> datasetFuture = datasetServiceClient.createDatasetAsync(locationName, dataset);
        System.out.format("Operation name: %s\n", datasetFuture.getInitialFuture().get().getName());
        System.out.println("Waiting for operation to finish...");
        Dataset datasetResponse = datasetFuture.get(300, TimeUnit.SECONDS);
        System.out.println("Create Dataset Table GCS sample");
        System.out.format("Name: %s\n", datasetResponse.getName());
        System.out.format("Display Name: %s\n", datasetResponse.getDisplayName());
        System.out.format("Metadata Schema Uri: %s\n", datasetResponse.getMetadataSchemaUri());
        System.out.format("Metadata: %s\n", datasetResponse.getMetadata());
    }
}
Also used : DatasetServiceSettings(com.google.cloud.aiplatform.v1.DatasetServiceSettings) CreateDatasetOperationMetadata(com.google.cloud.aiplatform.v1.CreateDatasetOperationMetadata) Dataset(com.google.cloud.aiplatform.v1.Dataset) DatasetServiceClient(com.google.cloud.aiplatform.v1.DatasetServiceClient) Value(com.google.protobuf.Value) LocationName(com.google.cloud.aiplatform.v1.LocationName)

Example 14 with DatasetServiceSettings

use of com.google.cloud.aiplatform.v1.DatasetServiceSettings in project java-aiplatform by googleapis.

the class ImportDataImageClassificationSampleTest method tearDown.

@After
public void tearDown() throws InterruptedException, ExecutionException, IOException {
    // delete the temp dataset
    DatasetServiceSettings datasetServiceSettings = DatasetServiceSettings.newBuilder().setEndpoint("us-central1-aiplatform.googleapis.com:443").build();
    try (DatasetServiceClient datasetServiceClient = DatasetServiceClient.create(datasetServiceSettings)) {
        DatasetName datasetName = DatasetName.of(PROJECT, LOCATION, datasetId);
        OperationFuture<Empty, DeleteOperationMetadata> operationFuture = datasetServiceClient.deleteDatasetAsync(datasetName);
        operationFuture.get();
    }
    System.out.flush();
    System.setOut(originalPrintStream);
}
Also used : Empty(com.google.protobuf.Empty) DatasetServiceSettings(com.google.cloud.aiplatform.v1beta1.DatasetServiceSettings) DatasetName(com.google.cloud.aiplatform.v1beta1.DatasetName) DeleteOperationMetadata(com.google.cloud.aiplatform.v1beta1.DeleteOperationMetadata) DatasetServiceClient(com.google.cloud.aiplatform.v1beta1.DatasetServiceClient) After(org.junit.After)

Example 15 with DatasetServiceSettings

use of com.google.cloud.aiplatform.v1.DatasetServiceSettings in project java-aiplatform by googleapis.

the class ImportDataImageClassificationSampleTest method setUp.

@Before
public void setUp() throws IOException, InterruptedException, ExecutionException, TimeoutException {
    bout = new ByteArrayOutputStream();
    out = new PrintStream(bout);
    originalPrintStream = System.out;
    System.setOut(out);
    // create a temp dataset for importing data
    DatasetServiceSettings datasetServiceSettings = DatasetServiceSettings.newBuilder().setEndpoint("us-central1-aiplatform.googleapis.com:443").build();
    try (DatasetServiceClient datasetServiceClient = DatasetServiceClient.create(datasetServiceSettings)) {
        String metadataSchemaUri = "gs://google-cloud-aiplatform/schema/dataset/metadata/image_1.0.0.yaml";
        LocationName locationName = LocationName.of(PROJECT, LOCATION);
        Dataset dataset = Dataset.newBuilder().setDisplayName("test_dataset_display_name").setMetadataSchemaUri(metadataSchemaUri).build();
        OperationFuture<Dataset, CreateDatasetOperationMetadata> datasetFuture = datasetServiceClient.createDatasetAsync(locationName, dataset);
        Dataset datasetResponse = datasetFuture.get(120, TimeUnit.SECONDS);
        String[] datasetValues = datasetResponse.getName().split("/");
        datasetId = datasetValues[datasetValues.length - 1];
    }
}
Also used : PrintStream(java.io.PrintStream) DatasetServiceSettings(com.google.cloud.aiplatform.v1beta1.DatasetServiceSettings) CreateDatasetOperationMetadata(com.google.cloud.aiplatform.v1beta1.CreateDatasetOperationMetadata) Dataset(com.google.cloud.aiplatform.v1beta1.Dataset) DatasetServiceClient(com.google.cloud.aiplatform.v1beta1.DatasetServiceClient) ByteArrayOutputStream(java.io.ByteArrayOutputStream) LocationName(com.google.cloud.aiplatform.v1beta1.LocationName) Before(org.junit.Before)

Aggregations

DatasetServiceClient (com.google.cloud.aiplatform.v1.DatasetServiceClient)15 DatasetServiceSettings (com.google.cloud.aiplatform.v1.DatasetServiceSettings)15 DatasetServiceClient (com.google.cloud.aiplatform.v1beta1.DatasetServiceClient)10 DatasetServiceSettings (com.google.cloud.aiplatform.v1beta1.DatasetServiceSettings)10 DatasetName (com.google.cloud.aiplatform.v1.DatasetName)9 GcsSource (com.google.cloud.aiplatform.v1.GcsSource)8 ImportDataConfig (com.google.cloud.aiplatform.v1.ImportDataConfig)8 ImportDataOperationMetadata (com.google.cloud.aiplatform.v1.ImportDataOperationMetadata)8 ImportDataResponse (com.google.cloud.aiplatform.v1.ImportDataResponse)8 CreateDatasetOperationMetadata (com.google.cloud.aiplatform.v1.CreateDatasetOperationMetadata)6 Dataset (com.google.cloud.aiplatform.v1.Dataset)6 LocationName (com.google.cloud.aiplatform.v1.LocationName)6 Empty (com.google.protobuf.Empty)6 CreateDatasetOperationMetadata (com.google.cloud.aiplatform.v1beta1.CreateDatasetOperationMetadata)5 Dataset (com.google.cloud.aiplatform.v1beta1.Dataset)5 DatasetName (com.google.cloud.aiplatform.v1beta1.DatasetName)5 DeleteOperationMetadata (com.google.cloud.aiplatform.v1beta1.DeleteOperationMetadata)5 LocationName (com.google.cloud.aiplatform.v1beta1.LocationName)5 ByteArrayOutputStream (java.io.ByteArrayOutputStream)5 PrintStream (java.io.PrintStream)5