Search in sources :

Example 76 with LocationName

use of com.google.cloud.aiplatform.v1.LocationName in project java-automl by googleapis.

the class ListDatasets method listDatasets.

// List the datasets
static void listDatasets(String projectId) throws IOException {
    // the "close" method on the client to safely clean up any remaining background resources.
    try (AutoMlClient client = AutoMlClient.create()) {
        // A resource that represents Google Cloud Platform location.
        LocationName projectLocation = LocationName.of(projectId, "us-central1");
        ListDatasetsRequest request = ListDatasetsRequest.newBuilder().setParent(projectLocation.toString()).build();
        // List all the datasets available in the region by applying filter.
        System.out.println("List of datasets:");
        for (Dataset dataset : client.listDatasets(request).iterateAll()) {
            // Display the dataset information
            System.out.format("\nDataset name: %s\n", dataset.getName());
            // To get the dataset id, you have to parse it out of the `name` field. As dataset Ids are
            // required for other methods.
            // Name Form: `projects/{project_id}/locations/{location_id}/datasets/{dataset_id}`
            String[] names = dataset.getName().split("/");
            String retrievedDatasetId = names[names.length - 1];
            System.out.format("Dataset id: %s\n", retrievedDatasetId);
            System.out.format("Dataset display name: %s\n", dataset.getDisplayName());
            System.out.println("Dataset create time:");
            System.out.format("\tseconds: %s\n", dataset.getCreateTime().getSeconds());
            System.out.format("\tnanos: %s\n", dataset.getCreateTime().getNanos());
            // [END automl_language_sentiment_analysis_list_datasets]
            // [END automl_language_text_classification_list_datasets]
            // [END automl_translate_list_datasets]
            // [END automl_vision_classification_list_datasets]
            // [END automl_vision_object_detection_list_datasets]
            System.out.format("Text extraction dataset metadata: %s\n", dataset.getTextExtractionDatasetMetadata());
            // [END automl_language_entity_extraction_list_datasets]
            // [START automl_language_sentiment_analysis_list_datasets]
            System.out.format("Text sentiment dataset metadata: %s\n", dataset.getTextSentimentDatasetMetadata());
            // [END automl_language_sentiment_analysis_list_datasets]
            // [START automl_language_text_classification_list_datasets]
            System.out.format("Text classification dataset metadata: %s\n", dataset.getTextClassificationDatasetMetadata());
            // [END automl_language_text_classification_list_datasets]
            // [START automl_translate_list_datasets]
            System.out.println("Translation dataset metadata:");
            System.out.format("\tSource language code: %s\n", dataset.getTranslationDatasetMetadata().getSourceLanguageCode());
            System.out.format("\tTarget language code: %s\n", dataset.getTranslationDatasetMetadata().getTargetLanguageCode());
            // [END automl_translate_list_datasets]
            // [START automl_vision_classification_list_datasets]
            System.out.format("Image classification dataset metadata: %s\n", dataset.getImageClassificationDatasetMetadata());
            // [END automl_vision_classification_list_datasets]
            // [START automl_vision_object_detection_list_datasets]
            System.out.format("Image object detection dataset metadata: %s\n", dataset.getImageObjectDetectionDatasetMetadata());
        // [START automl_language_entity_extraction_list_datasets]
        // [START automl_language_sentiment_analysis_list_datasets]
        // [START automl_language_text_classification_list_datasets]
        // [START automl_translate_list_datasets]
        // [START automl_vision_classification_list_datasets]
        }
    }
}
Also used : Dataset(com.google.cloud.automl.v1.Dataset) AutoMlClient(com.google.cloud.automl.v1.AutoMlClient) LocationName(com.google.cloud.automl.v1.LocationName) ListDatasetsRequest(com.google.cloud.automl.v1.ListDatasetsRequest)

Example 77 with LocationName

use of com.google.cloud.aiplatform.v1.LocationName in project java-automl by googleapis.

the class TranslateCreateDataset method createDataset.

// Create a dataset
static void createDataset(String projectId, String displayName) throws IOException, ExecutionException, InterruptedException {
    // the "close" method on the client to safely clean up any remaining background resources.
    try (AutoMlClient client = AutoMlClient.create()) {
        // A resource that represents Google Cloud Platform location.
        LocationName projectLocation = LocationName.of(projectId, "us-central1");
        // Specify the source and target language.
        TranslationDatasetMetadata translationDatasetMetadata = TranslationDatasetMetadata.newBuilder().setSourceLanguageCode("en").setTargetLanguageCode("ja").build();
        Dataset dataset = Dataset.newBuilder().setDisplayName(displayName).setTranslationDatasetMetadata(translationDatasetMetadata).build();
        OperationFuture<Dataset, OperationMetadata> future = client.createDatasetAsync(projectLocation, dataset);
        Dataset createdDataset = future.get();
        // Display the dataset information.
        System.out.format("Dataset name: %s\n", createdDataset.getName());
        // To get the dataset id, you have to parse it out of the `name` field. As dataset Ids are
        // required for other methods.
        // Name Form: `projects/{project_id}/locations/{location_id}/datasets/{dataset_id}`
        String[] names = createdDataset.getName().split("/");
        String datasetId = names[names.length - 1];
        System.out.format("Dataset id: %s\n", datasetId);
    }
}
Also used : TranslationDatasetMetadata(com.google.cloud.automl.v1.TranslationDatasetMetadata) Dataset(com.google.cloud.automl.v1.Dataset) OperationMetadata(com.google.cloud.automl.v1.OperationMetadata) AutoMlClient(com.google.cloud.automl.v1.AutoMlClient) LocationName(com.google.cloud.automl.v1.LocationName)

Example 78 with LocationName

use of com.google.cloud.aiplatform.v1.LocationName in project java-automl by googleapis.

the class TranslateCreateModel method createModel.

// Create a model
static void createModel(String projectId, String datasetId, String displayName) throws IOException, ExecutionException, InterruptedException {
    // the "close" method on the client to safely clean up any remaining background resources.
    try (AutoMlClient client = AutoMlClient.create()) {
        // A resource that represents Google Cloud Platform location.
        LocationName projectLocation = LocationName.of(projectId, "us-central1");
        TranslationModelMetadata translationModelMetadata = TranslationModelMetadata.newBuilder().build();
        Model model = Model.newBuilder().setDisplayName(displayName).setDatasetId(datasetId).setTranslationModelMetadata(translationModelMetadata).build();
        // Create a model with the model metadata in the region.
        OperationFuture<Model, OperationMetadata> future = client.createModelAsync(projectLocation, model);
        // OperationFuture.get() will block until the model is created, which may take several hours.
        // You can use OperationFuture.getInitialFuture to get a future representing the initial
        // response to the request, which contains information while the operation is in progress.
        System.out.format("Training operation name: %s\n", future.getInitialFuture().get().getName());
        System.out.println("Training started...");
    }
}
Also used : Model(com.google.cloud.automl.v1.Model) TranslationModelMetadata(com.google.cloud.automl.v1.TranslationModelMetadata) OperationMetadata(com.google.cloud.automl.v1.OperationMetadata) AutoMlClient(com.google.cloud.automl.v1.AutoMlClient) LocationName(com.google.cloud.automl.v1.LocationName)

Example 79 with LocationName

use of com.google.cloud.aiplatform.v1.LocationName in project java-automl by googleapis.

the class GetOperationStatusTest method setUp.

@Before
public void setUp() throws IOException {
    // Use list operations to get a single operation id for the get call.
    try (AutoMlClient client = AutoMlClient.create()) {
        LocationName projectLocation = LocationName.of(PROJECT_ID, "us-central1");
        ListOperationsRequest request = ListOperationsRequest.newBuilder().setName(projectLocation.toString()).build();
        Operation operation = client.getOperationsClient().listOperations(request).iterateAll().iterator().next();
        operationId = operation.getName();
    }
    bout = new ByteArrayOutputStream();
    out = new PrintStream(bout);
    originalPrintStream = System.out;
    System.setOut(out);
}
Also used : PrintStream(java.io.PrintStream) ListOperationsRequest(com.google.longrunning.ListOperationsRequest) Operation(com.google.longrunning.Operation) ByteArrayOutputStream(java.io.ByteArrayOutputStream) AutoMlClient(com.google.cloud.automl.v1beta1.AutoMlClient) LocationName(com.google.cloud.automl.v1beta1.LocationName) Before(org.junit.Before)

Example 80 with LocationName

use of com.google.cloud.aiplatform.v1.LocationName in project java-automl by googleapis.

the class ListOperationStatusTest method setUp.

@Before
public void setUp() throws IOException, InterruptedException {
    bout = new ByteArrayOutputStream();
    out = new PrintStream(bout);
    originalPrintStream = System.out;
    System.setOut(out);
    // if the LRO status count more than 300, delete half of operations.
    try (AutoMlClient client = AutoMlClient.create()) {
        OperationsClient operationsClient = client.getOperationsClient();
        LocationName projectLocation = LocationName.of(PROJECT_ID, "us-central1");
        ListOperationsRequest listRequest = ListOperationsRequest.newBuilder().setName(projectLocation.toString()).build();
        List<String> operationFullPathsToBeDeleted = new ArrayList<>();
        for (Operation operation : operationsClient.listOperations(listRequest).iterateAll()) {
            // Filter: deleting already done operations.
            if (operation.getDone() && !operation.hasError()) {
                operationFullPathsToBeDeleted.add(operation.getName());
            }
        }
        if (operationFullPathsToBeDeleted.size() > 300) {
            System.out.println("Cleaning up...");
            for (String operationFullPath : operationFullPathsToBeDeleted.subList(0, operationFullPathsToBeDeleted.size() / 2)) {
                // retry_interval * (random value in range [1 - rand_factor, 1 + rand_factor])
                ExponentialBackOff exponentialBackOff = new ExponentialBackOff.Builder().setInitialIntervalMillis(60000).setMaxElapsedTimeMillis(300000).setRandomizationFactor(0.5).setMultiplier(1.1).setMaxIntervalMillis(80000).build();
                // delete unused operations.
                try {
                    operationsClient.deleteOperation(operationFullPath);
                } catch (ResourceExhaustedException ex) {
                    // exponential back off and retry.
                    long backOffInMillis = exponentialBackOff.nextBackOffMillis();
                    System.out.printf("Backing off for %d milliseconds " + "due to Resource exhaustion.\n", backOffInMillis);
                    if (backOffInMillis < 0) {
                        break;
                    }
                    System.out.println("Backing off" + backOffInMillis);
                    TimeUnit.MILLISECONDS.sleep(backOffInMillis);
                } catch (Exception ex) {
                    throw ex;
                }
            }
        } else {
            // Clear the list since we wont anything with the list.
            operationFullPathsToBeDeleted.clear();
        }
    }
}
Also used : PrintStream(java.io.PrintStream) ArrayList(java.util.ArrayList) OperationsClient(com.google.longrunning.OperationsClient) ByteArrayOutputStream(java.io.ByteArrayOutputStream) Operation(com.google.longrunning.Operation) ExponentialBackOff(com.google.api.client.util.ExponentialBackOff) IOException(java.io.IOException) ResourceExhaustedException(com.google.api.gax.rpc.ResourceExhaustedException) LocationName(com.google.cloud.automl.v1.LocationName) ResourceExhaustedException(com.google.api.gax.rpc.ResourceExhaustedException) ListOperationsRequest(com.google.longrunning.ListOperationsRequest) AutoMlClient(com.google.cloud.automl.v1.AutoMlClient) Before(org.junit.Before)

Aggregations

Test (org.junit.Test)54 AbstractMessage (com.google.protobuf.AbstractMessage)37 LocationName (com.google.cloud.aiplatform.v1.LocationName)36 LocationName (com.google.privacy.dlp.v2.LocationName)22 OrganizationLocationName (com.google.privacy.dlp.v2.OrganizationLocationName)22 LocationName (com.google.cloud.translate.v3beta1.LocationName)18 TranslationServiceClient (com.google.cloud.translate.v3beta1.TranslationServiceClient)18 AutoMlClient (com.google.cloud.automl.v1.AutoMlClient)17 LocationName (com.google.cloud.automl.v1.LocationName)17 JobServiceClient (com.google.cloud.aiplatform.v1.JobServiceClient)15 JobServiceSettings (com.google.cloud.aiplatform.v1.JobServiceSettings)15 Value (com.google.protobuf.Value)15 InvalidArgumentException (com.google.api.gax.rpc.InvalidArgumentException)14 Model (com.google.cloud.aiplatform.v1.Model)14 StatusRuntimeException (io.grpc.StatusRuntimeException)14 PipelineServiceClient (com.google.cloud.aiplatform.v1.PipelineServiceClient)13 PipelineServiceSettings (com.google.cloud.aiplatform.v1.PipelineServiceSettings)13 TrainingPipeline (com.google.cloud.aiplatform.v1.TrainingPipeline)13 LocationName (com.google.cloud.translate.v3.LocationName)13 TranslationServiceClient (com.google.cloud.translate.v3.TranslationServiceClient)13