Examples with StorageObject com.google.api.services.storage.model.StorageObject used on opensource projects

Example 36 with StorageObject

use of com.google.api.services.storage.model.StorageObject in project beam by apache.

the class GcsUtilTest method testRetryFileSizeNonBatch.

@Test
public void testRetryFileSizeNonBatch() throws IOException {
    GcsOptions pipelineOptions = gcsOptionsWithTestCredential();
    GcsUtil gcsUtil = pipelineOptions.getGcsUtil();
    Storage mockStorage = Mockito.mock(Storage.class);
    gcsUtil.setStorageClient(mockStorage);
    Storage.Objects mockStorageObjects = Mockito.mock(Storage.Objects.class);
    Storage.Objects.Get mockStorageGet = Mockito.mock(Storage.Objects.Get.class);
    BackOff mockBackOff = BackOffAdapter.toGcpBackOff(FluentBackoff.DEFAULT.withMaxRetries(2).backoff());
    when(mockStorage.objects()).thenReturn(mockStorageObjects);
    when(mockStorageObjects.get("testbucket", "testobject")).thenReturn(mockStorageGet);
    when(mockStorageGet.execute()).thenThrow(new SocketTimeoutException("SocketException")).thenThrow(new SocketTimeoutException("SocketException")).thenReturn(new StorageObject().setSize(BigInteger.valueOf(1000)));
    assertEquals(1000, gcsUtil.getObject(GcsPath.fromComponents("testbucket", "testobject"), mockBackOff, new FastNanoClockAndSleeper()).getSize().longValue());
    assertEquals(BackOff.STOP, mockBackOff.nextBackOffMillis());
}

Example 37 with StorageObject

use of com.google.api.services.storage.model.StorageObject in project ignite by apache.

the class TcpDiscoveryGoogleStorageIpFinder method getRegisteredAddresses.

/**
 * {@inheritDoc}
 */
@Override
public Collection<InetSocketAddress> getRegisteredAddresses() throws IgniteSpiException {
    init();
    Collection<InetSocketAddress> addrs = new ArrayList<>();
    try {
        Storage.Objects.List listObjects = storage.objects().list(bucketName);
        com.google.api.services.storage.model.Objects objects;
        do {
            objects = listObjects.execute();
            if (objects == null || objects.getItems() == null)
                break;
            for (StorageObject object : objects.getItems()) addrs.add(addrFromString(object.getName()));
            listObjects.setPageToken(objects.getNextPageToken());
        } while (null != objects.getNextPageToken());
    } catch (Exception e) {
        throw new IgniteSpiException("Failed to get content from the bucket: " + bucketName, e);
    }
    return addrs;
}

Example 38 with StorageObject

use of com.google.api.services.storage.model.StorageObject in project gradle by gradle.

the class GcsClient method put.

public void put(InputStream inputStream, Long contentLength, URI destination) throws ResourceException {
    try {
        InputStreamContent contentStream = new InputStreamContent(null, inputStream);
        // Setting the length improves upload performance
        contentStream.setLength(contentLength);
        // TODO - set ACL here if necessary
        String bucket = destination.getHost();
        String path = cleanResourcePath(destination);
        StorageObject objectMetadata = new StorageObject().setName(path);
        Storage.Objects.Insert putRequest = storage.objects().insert(bucket, objectMetadata, contentStream);
        LOGGER.debug("Attempting to put resource:[{}] into gcs bucket [{}]", putRequest.getName(), putRequest.getBucket());
        putRequest.execute();
    } catch (IOException e) {
        throw ResourceExceptions.putFailed(destination, e);
    }
}

Example 39 with StorageObject

use of com.google.api.services.storage.model.StorageObject in project gradle by gradle.

the class GcsResourceConnector method getMetaData.

@Nullable
@Override
public ExternalResourceMetaData getMetaData(URI location, boolean revalidate) throws ResourceException {
    LOGGER.debug("Attempting to get resource metadata: {}", location);
    StorageObject gcsObject = gcsClient.getResource(location);
    if (gcsObject == null) {
        return null;
    }
    return toExternalResourceMetaData(location, gcsObject);
}