Search in sources :

Example 1 with Field

use of com.google.cloud.bigquery.Field in project google-cloud-java by GoogleCloudPlatform.

the class CreateTableAndLoadData method main.

public static void main(String... args) throws InterruptedException, TimeoutException {
    BigQuery bigquery = BigQueryOptions.getDefaultInstance().getService();
    TableId tableId = TableId.of("dataset", "table");
    Table table = bigquery.getTable(tableId);
    if (table == null) {
        System.out.println("Creating table " + tableId);
        Field integerField = Field.of("fieldName", Field.Type.integer());
        Schema schema = Schema.of(integerField);
        table = bigquery.create(TableInfo.of(tableId, StandardTableDefinition.of(schema)));
    }
    System.out.println("Loading data into table " + tableId);
    Job loadJob = table.load(FormatOptions.csv(), "gs://bucket/path");
    loadJob = loadJob.waitFor();
    if (loadJob.getStatus().getError() != null) {
        System.out.println("Job completed with errors");
    } else {
        System.out.println("Job succeeded");
    }
}
Also used : TableId(com.google.cloud.bigquery.TableId) Field(com.google.cloud.bigquery.Field) BigQuery(com.google.cloud.bigquery.BigQuery) Table(com.google.cloud.bigquery.Table) Schema(com.google.cloud.bigquery.Schema) Job(com.google.cloud.bigquery.Job)

Example 2 with Field

use of com.google.cloud.bigquery.Field in project google-cloud-java by GoogleCloudPlatform.

the class BigQuerySnippets method createTable.

/**
   * Example of creating a table.
   */
// [TARGET create(TableInfo, TableOption...)]
// [VARIABLE "my_dataset_name"]
// [VARIABLE "my_table_name"]
// [VARIABLE "string_field"]
public Table createTable(String datasetName, String tableName, String fieldName) {
    // [START createTable]
    TableId tableId = TableId.of(datasetName, tableName);
    // Table field definition
    Field field = Field.of(fieldName, Field.Type.string());
    // Table schema definition
    Schema schema = Schema.of(field);
    TableDefinition tableDefinition = StandardTableDefinition.of(schema);
    TableInfo tableInfo = TableInfo.newBuilder(tableId, tableDefinition).build();
    Table table = bigquery.create(tableInfo);
    // [END createTable]
    return table;
}
Also used : TableId(com.google.cloud.bigquery.TableId) Field(com.google.cloud.bigquery.Field) Table(com.google.cloud.bigquery.Table) Schema(com.google.cloud.bigquery.Schema) StandardTableDefinition(com.google.cloud.bigquery.StandardTableDefinition) TableDefinition(com.google.cloud.bigquery.TableDefinition) TableInfo(com.google.cloud.bigquery.TableInfo)

Example 3 with Field

use of com.google.cloud.bigquery.Field in project google-cloud-java by GoogleCloudPlatform.

the class InsertDataAndQueryTable method main.

public static void main(String... args) throws InterruptedException {
    // Create a service instance
    BigQuery bigquery = BigQueryOptions.getDefaultInstance().getService();
    // Create a dataset
    String datasetId = "my_dataset_id";
    bigquery.create(DatasetInfo.newBuilder(datasetId).build());
    TableId tableId = TableId.of(datasetId, "my_table_id");
    // Table field definition
    Field stringField = Field.of("StringField", Field.Type.string());
    // Table schema definition
    Schema schema = Schema.of(stringField);
    // Create a table
    StandardTableDefinition tableDefinition = StandardTableDefinition.of(schema);
    bigquery.create(TableInfo.of(tableId, tableDefinition));
    // Define rows to insert
    Map<String, Object> firstRow = new HashMap<>();
    Map<String, Object> secondRow = new HashMap<>();
    firstRow.put("StringField", "value1");
    secondRow.put("StringField", "value2");
    // Create an insert request
    InsertAllRequest insertRequest = InsertAllRequest.newBuilder(tableId).addRow(firstRow).addRow(secondRow).build();
    // Insert rows
    InsertAllResponse insertResponse = bigquery.insertAll(insertRequest);
    // Check if errors occurred
    if (insertResponse.hasErrors()) {
        System.out.println("Errors occurred while inserting rows");
    }
    // Create a query request
    QueryRequest queryRequest = QueryRequest.newBuilder("SELECT * FROM my_dataset_id.my_table_id").setMaxWaitTime(60000L).setPageSize(1000L).build();
    // Request query to be executed and wait for results
    QueryResponse queryResponse = bigquery.query(queryRequest);
    while (!queryResponse.jobCompleted()) {
        Thread.sleep(1000L);
        queryResponse = bigquery.getQueryResults(queryResponse.getJobId());
    }
    // Read rows
    System.out.println("Table rows:");
    for (List<FieldValue> row : queryResponse.getResult().iterateAll()) {
        System.out.println(row);
    }
}
Also used : TableId(com.google.cloud.bigquery.TableId) BigQuery(com.google.cloud.bigquery.BigQuery) QueryRequest(com.google.cloud.bigquery.QueryRequest) HashMap(java.util.HashMap) Schema(com.google.cloud.bigquery.Schema) InsertAllResponse(com.google.cloud.bigquery.InsertAllResponse) Field(com.google.cloud.bigquery.Field) QueryResponse(com.google.cloud.bigquery.QueryResponse) FieldValue(com.google.cloud.bigquery.FieldValue) StandardTableDefinition(com.google.cloud.bigquery.StandardTableDefinition) InsertAllRequest(com.google.cloud.bigquery.InsertAllRequest)

Example 4 with Field

use of com.google.cloud.bigquery.Field in project google-cloud-java by GoogleCloudPlatform.

the class ITDatasetSnippets method testCreateTable.

@Test
public void testCreateTable() {
    String expectedTableName = "test_table";
    String expectedFieldName = "test_field";
    Table actualTable = datasetSnippets.createTable(expectedTableName, expectedFieldName);
    assertNotNull(actualTable);
    assertEquals(expectedTableName, actualTable.getTableId().getTable());
    assertEquals(1, actualTable.getDefinition().getSchema().getFields().size());
    Field actualField = actualTable.getDefinition().getSchema().getFields().get(0);
    assertEquals(expectedFieldName, actualField.getName());
    bigquery.delete(DATASET, expectedTableName);
}
Also used : Field(com.google.cloud.bigquery.Field) Table(com.google.cloud.bigquery.Table) Test(org.junit.Test)

Aggregations

Field (com.google.cloud.bigquery.Field)4 Schema (com.google.cloud.bigquery.Schema)3 Table (com.google.cloud.bigquery.Table)3 TableId (com.google.cloud.bigquery.TableId)3 BigQuery (com.google.cloud.bigquery.BigQuery)2 StandardTableDefinition (com.google.cloud.bigquery.StandardTableDefinition)2 FieldValue (com.google.cloud.bigquery.FieldValue)1 InsertAllRequest (com.google.cloud.bigquery.InsertAllRequest)1 InsertAllResponse (com.google.cloud.bigquery.InsertAllResponse)1 Job (com.google.cloud.bigquery.Job)1 QueryRequest (com.google.cloud.bigquery.QueryRequest)1 QueryResponse (com.google.cloud.bigquery.QueryResponse)1 TableDefinition (com.google.cloud.bigquery.TableDefinition)1 TableInfo (com.google.cloud.bigquery.TableInfo)1 HashMap (java.util.HashMap)1 Test (org.junit.Test)1