Examples with JobConfigurationExtract com.google.api.services.bigquery.model.JobConfigurationExtract used on opensource projects

Example 1 with JobConfigurationExtract

use of com.google.api.services.bigquery.model.JobConfigurationExtract in project beam by apache.

the class BigQuerySourceBase method executeExtract.

private List<ResourceId> executeExtract(String jobId, TableReference table, JobService jobService, String executingProject, String extractDestinationDir) throws InterruptedException, IOException {
    JobReference jobRef = new JobReference().setProjectId(executingProject).setJobId(jobId);
    String destinationUri = BigQueryIO.getExtractDestinationUri(extractDestinationDir);
    JobConfigurationExtract extract = new JobConfigurationExtract().setSourceTable(table).setDestinationFormat("AVRO").setDestinationUris(ImmutableList.of(destinationUri));
    LOG.info("Starting BigQuery extract job: {}", jobId);
    jobService.startExtractJob(jobRef, extract);
    Job extractJob = jobService.pollJob(jobRef, JOB_POLL_MAX_RETRIES);
    if (BigQueryHelpers.parseStatus(extractJob) != Status.SUCCEEDED) {
        throw new IOException(String.format("Extract job %s failed, status: %s.", extractJob.getJobReference().getJobId(), BigQueryHelpers.statusToPrettyString(extractJob.getStatus())));
    }
    LOG.info("BigQuery extract job completed: {}", jobId);
    return BigQueryIO.getExtractFilePaths(extractDestinationDir, extractJob);
}

Example 2 with JobConfigurationExtract

use of com.google.api.services.bigquery.model.JobConfigurationExtract in project google-cloud-java by GoogleCloudPlatform.

the class ExtractJobConfiguration method toPb.

@Override
com.google.api.services.bigquery.model.JobConfiguration toPb() {
    JobConfigurationExtract extractConfigurationPb = new JobConfigurationExtract();
    extractConfigurationPb.setDestinationUris(destinationUris);
    extractConfigurationPb.setSourceTable(sourceTable.toPb());
    extractConfigurationPb.setPrintHeader(printHeader);
    extractConfigurationPb.setFieldDelimiter(fieldDelimiter);
    extractConfigurationPb.setDestinationFormat(format);
    extractConfigurationPb.setCompression(compression);
    return new com.google.api.services.bigquery.model.JobConfiguration().setExtract(extractConfigurationPb);
}