Search in sources :

Example 16 with ContentCacheKey

use of org.talend.dataprep.cache.ContentCacheKey in project data-prep by Talend.

the class ResourceLoaderContentCache method evictMatch.

@Timed
@Override
public void evictMatch(ContentCacheKey key) {
    try {
        final DeletableResource[] resources = resolver.getResources("/cache/" + key.getPrefix() + "**");
        final Predicate<String> matcher = key.getMatcher();
        stream(resources).filter(r -> matcher.test(r.getFilename())).forEach(r -> {
            try {
                r.delete();
            } catch (IOException e) {
                throw new TDPException(CommonErrorCodes.UNEXPECTED_EXCEPTION, e);
            }
        });
    } catch (IOException e) {
        throw new TDPException(CommonErrorCodes.UNEXPECTED_EXCEPTION, e);
    }
}
Also used : VolumeMetered(org.talend.dataprep.metrics.VolumeMetered) StringUtils(org.apache.commons.lang.StringUtils) TDPException(org.talend.dataprep.exception.TDPException) ContentCacheKey(org.talend.dataprep.cache.ContentCacheKey) LoggerFactory(org.slf4j.LoggerFactory) Autowired(org.springframework.beans.factory.annotation.Autowired) BinaryOperator.maxBy(java.util.function.BinaryOperator.maxBy) OutputStream(java.io.OutputStream) ConditionalOnBean(org.springframework.boot.autoconfigure.condition.ConditionalOnBean) StringUtils.substringAfterLast(org.apache.commons.lang.StringUtils.substringAfterLast) Logger(org.slf4j.Logger) DeletableResource(org.talend.daikon.content.DeletableResource) Optional.ofNullable(java.util.Optional.ofNullable) Predicate(java.util.function.Predicate) IOException(java.io.IOException) ContentCache(org.talend.dataprep.cache.ContentCache) Component(org.springframework.stereotype.Component) ContentServiceEnabled(org.talend.daikon.content.ContentServiceEnabled) Optional(java.util.Optional) CommonErrorCodes(org.talend.dataprep.exception.error.CommonErrorCodes) ResourceResolver(org.talend.daikon.content.ResourceResolver) Long.parseLong(java.lang.Long.parseLong) Arrays.stream(java.util.Arrays.stream) Timed(org.talend.dataprep.metrics.Timed) InputStream(java.io.InputStream) TDPException(org.talend.dataprep.exception.TDPException) DeletableResource(org.talend.daikon.content.DeletableResource) IOException(java.io.IOException) Timed(org.talend.dataprep.metrics.Timed)

Example 17 with ContentCacheKey

use of org.talend.dataprep.cache.ContentCacheKey in project data-prep by Talend.

the class TransformationCacheKeyTest method getKey_should_generate_serialized_key.

@Test
public void getKey_should_generate_serialized_key() throws Exception {
    // given
    final ContentCacheKey key = new TransformationCacheKey("prep1", "dataset1", "JSON", "step1", "param1", HEAD, "user1", "");
    // when
    final String keyStr = key.getKey();
    // then
    assertThat(keyStr, is("transformation_prep1_dataset1_b6aa01425c31e1eed71d0c3cbc7763aad865d1b1"));
}
Also used : TransformationCacheKey(org.talend.dataprep.cache.TransformationCacheKey) ContentCacheKey(org.talend.dataprep.cache.ContentCacheKey) Test(org.junit.Test)

Example 18 with ContentCacheKey

use of org.talend.dataprep.cache.ContentCacheKey in project data-prep by Talend.

the class TransformationCacheKeyTest method getMatcher_should_return_matcher_for_partial_key.

@Test
public void getMatcher_should_return_matcher_for_partial_key() throws Exception {
    // given
    final ContentCacheKey prepKey = new TransformationCacheKey("prep1", null, null, null, null, null, null, "");
    final ContentCacheKey dataSetKey = new TransformationCacheKey(null, "dataset1", null, null, null, null, null, "");
    final ContentCacheKey matchingKey = new TransformationCacheKey("prep1", "dataset1", "JSON", "step1", "param1", HEAD, "user1", "");
    final ContentCacheKey nonMatchingKey = new TransformationCacheKey("prep2", "dataset2", "XLS", "step2", "param2", FILTER, "user2", "");
    // when / then
    assertThat(prepKey.getMatcher().test(matchingKey.getKey()), is(true));
    assertThat(dataSetKey.getMatcher().test(matchingKey.getKey()), is(true));
    assertThat(prepKey.getMatcher().test(nonMatchingKey.getKey()), is(false));
    assertThat(dataSetKey.getMatcher().test(nonMatchingKey.getKey()), is(false));
}
Also used : TransformationCacheKey(org.talend.dataprep.cache.TransformationCacheKey) ContentCacheKey(org.talend.dataprep.cache.ContentCacheKey) Test(org.junit.Test)

Example 19 with ContentCacheKey

use of org.talend.dataprep.cache.ContentCacheKey in project data-prep by Talend.

the class TransformationMetadataCacheKeyTest method getKey_should_generate_serialized_key.

@Test
public void getKey_should_generate_serialized_key() throws Exception {
    // given
    final ContentCacheKey key = new TransformationMetadataCacheKey("prep1", "step1", HEAD, "user1");
    // when
    final String keyStr = key.getKey();
    // then
    assertThat(keyStr, is("transformation-metadata_prep1_step1_HEAD_user1"));
}
Also used : ContentCacheKey(org.talend.dataprep.cache.ContentCacheKey) TransformationMetadataCacheKey(org.talend.dataprep.cache.TransformationMetadataCacheKey) Test(org.junit.Test)

Example 20 with ContentCacheKey

use of org.talend.dataprep.cache.ContentCacheKey in project data-prep by Talend.

the class TransformationMetadataCacheKeyTest method getMatcher_should_return_matcher_for_partial_key.

@Test
public void getMatcher_should_return_matcher_for_partial_key() throws Exception {
    // given
    final ContentCacheKey prepKey = new TransformationMetadataCacheKey("prep1", null, null, null);
    final ContentCacheKey stepKey = new TransformationMetadataCacheKey(null, "step1", null, null);
    final ContentCacheKey sourceKey = new TransformationMetadataCacheKey(null, null, HEAD, null);
    final ContentCacheKey userKey = new TransformationMetadataCacheKey(null, null, null, "user1");
    final ContentCacheKey matchingKey = new TransformationMetadataCacheKey("prep1", "step1", HEAD, "user1");
    final ContentCacheKey nonMatchingKey = new TransformationMetadataCacheKey("prep2", "step2", FILTER, "user2");
    // when / then
    assertThat(prepKey.getMatcher().test(matchingKey.getKey()), is(true));
    assertThat(stepKey.getMatcher().test(matchingKey.getKey()), is(true));
    assertThat(sourceKey.getMatcher().test(matchingKey.getKey()), is(true));
    assertThat(userKey.getMatcher().test(matchingKey.getKey()), is(true));
    assertThat(prepKey.getMatcher().test(nonMatchingKey.getKey()), is(false));
    assertThat(stepKey.getMatcher().test(nonMatchingKey.getKey()), is(false));
    assertThat(sourceKey.getMatcher().test(nonMatchingKey.getKey()), is(false));
    assertThat(userKey.getMatcher().test(nonMatchingKey.getKey()), is(false));
}
Also used : ContentCacheKey(org.talend.dataprep.cache.ContentCacheKey) TransformationMetadataCacheKey(org.talend.dataprep.cache.TransformationMetadataCacheKey) Test(org.junit.Test)

Aggregations

ContentCacheKey (org.talend.dataprep.cache.ContentCacheKey)28 Test (org.junit.Test)20 ServiceBaseTest (org.talend.ServiceBaseTest)14 TransformationMetadataCacheKey (org.talend.dataprep.cache.TransformationMetadataCacheKey)4 OutputStream (java.io.OutputStream)3 ArrayList (java.util.ArrayList)3 DataSetMetadata (org.talend.dataprep.api.dataset.DataSetMetadata)3 TDPException (org.talend.dataprep.exception.TDPException)3 ApiOperation (io.swagger.annotations.ApiOperation)2 InputStream (java.io.InputStream)2 Predicate (java.util.function.Predicate)2 Logger (org.slf4j.Logger)2 LoggerFactory (org.slf4j.LoggerFactory)2 Autowired (org.springframework.beans.factory.annotation.Autowired)2 Component (org.springframework.stereotype.Component)2 TransformationCacheKey (org.talend.dataprep.cache.TransformationCacheKey)2 Timed (org.talend.dataprep.metrics.Timed)2 JsonParser (com.fasterxml.jackson.core.JsonParser)1 IOException (java.io.IOException)1 Long.parseLong (java.lang.Long.parseLong)1