Search in sources :

Example 6 with EventMetadata

use of org.commonjava.maven.galley.event.EventMetadata in project indy by Commonjava.

the class DownloadManagerTest method downloadOnePOMFromSingleRepository.

@Test
public void downloadOnePOMFromSingleRepository() throws Exception {
    final String content = "This is a test";
    final String path = "/org/apache/maven/maven-model/3.0.3/maven-model-3.0.3.pom";
    final RemoteRepository repo = new RemoteRepository(MAVEN_PKG_KEY, "central", "http://repo.maven.apache.org/maven2");
    fixture.getTransport().registerDownload(new ConcreteResource(new RepositoryLocation(repo), path), new TestDownload(content.getBytes()));
    data.storeArtifactStore(repo, summary, false, true, new EventMetadata());
    final Transfer stream = downloader.retrieve(repo, path, new EventMetadata());
    final String downloaded = IOUtils.toString(stream.openInputStream());
    assertThat(downloaded, equalTo(content));
}
Also used : TestDownload(org.commonjava.maven.galley.testing.core.transport.job.TestDownload) ConcreteResource(org.commonjava.maven.galley.model.ConcreteResource) Transfer(org.commonjava.maven.galley.model.Transfer) RemoteRepository(org.commonjava.indy.model.core.RemoteRepository) RepositoryLocation(org.commonjava.indy.model.galley.RepositoryLocation) EventMetadata(org.commonjava.maven.galley.event.EventMetadata) Test(org.junit.Test)

Example 7 with EventMetadata

use of org.commonjava.maven.galley.event.EventMetadata in project indy by Commonjava.

the class DownloadManagerTest method downloadOnePOMFromSecondRepositoryAfterDummyRepoFails.

@Test
public void downloadOnePOMFromSecondRepositoryAfterDummyRepoFails() throws Exception {
    final RemoteRepository repo = new RemoteRepository(MAVEN_PKG_KEY, "dummy", "http://www.nowhere.com/");
    final String content = "This is a test";
    final String path = "/org/apache/maven/maven-model/3.0.3/maven-model-3.0.3.pom";
    final RemoteRepository repo2 = new RemoteRepository(MAVEN_PKG_KEY, "central", "http://repo.maven.apache.org/maven2");
    fixture.getTransport().registerDownload(new ConcreteResource(new RepositoryLocation(repo2), path), new TestDownload(content.getBytes()));
    data.storeArtifactStore(repo, summary, false, true, new EventMetadata());
    data.storeArtifactStore(repo2, summary, false, true, new EventMetadata());
    final List<ArtifactStore> repos = new ArrayList<ArtifactStore>();
    repos.add(repo);
    repos.add(repo2);
    final Transfer stream = downloader.retrieveFirst(repos, path, new EventMetadata());
    final String downloaded = IOUtils.toString(stream.openInputStream());
    assertThat(downloaded, equalTo(content));
}
Also used : TestDownload(org.commonjava.maven.galley.testing.core.transport.job.TestDownload) ArtifactStore(org.commonjava.indy.model.core.ArtifactStore) ConcreteResource(org.commonjava.maven.galley.model.ConcreteResource) ArrayList(java.util.ArrayList) Transfer(org.commonjava.maven.galley.model.Transfer) RemoteRepository(org.commonjava.indy.model.core.RemoteRepository) RepositoryLocation(org.commonjava.indy.model.galley.RepositoryLocation) EventMetadata(org.commonjava.maven.galley.event.EventMetadata) Test(org.junit.Test)

Example 8 with EventMetadata

use of org.commonjava.maven.galley.event.EventMetadata in project indy by Commonjava.

the class DataFileStoreDataManager method readDefinitions.

@PostConstruct
public void readDefinitions() {
    final ChangeSummary summary = new ChangeSummary(ChangeSummary.SYSTEM_USER, "Reading definitions from disk, culling invalid definition files.");
    try {
        DataFile[] packageDirs = manager.getDataFile(INDY_STORE).listFiles((f) -> true);
        for (DataFile pkgDir : packageDirs) {
            for (StoreType type : StoreType.values()) {
                DataFile[] files = pkgDir.getChild(type.singularEndpointName()).listFiles(f -> true);
                if (files != null) {
                    for (final DataFile f : files) {
                        try {
                            final String json = f.readString();
                            final ArtifactStore store = serializer.readValue(json, type.getStoreClass());
                            if (store == null) {
                                f.delete(summary);
                            } else {
                                storeArtifactStore(store, summary, false, false, new EventMetadata().set(StoreDataManager.EVENT_ORIGIN, LOAD_FROM_DISK));
                            }
                        } catch (final IOException e) {
                            logger.error(String.format("Failed to load %s store: %s. Reason: %s", type, f, e.getMessage()), e);
                        }
                    }
                }
            }
        }
        started = true;
    } catch (final IndyDataException e) {
        throw new IllegalStateException("Failed to start store data manager: " + e.getMessage(), e);
    }
}
Also used : DataFile(org.commonjava.indy.subsys.datafile.DataFile) StoreType(org.commonjava.indy.model.core.StoreType) IndyDataException(org.commonjava.indy.data.IndyDataException) ArtifactStore(org.commonjava.indy.model.core.ArtifactStore) IOException(java.io.IOException) ChangeSummary(org.commonjava.indy.audit.ChangeSummary) EventMetadata(org.commonjava.maven.galley.event.EventMetadata) PostConstruct(javax.annotation.PostConstruct)

Example 9 with EventMetadata

use of org.commonjava.maven.galley.event.EventMetadata in project indy by Commonjava.

the class ConcurrencyTest method deadlockOnGroupContains.

@BMRules(rules = { @BMRule(name = "init rendezvous", targetClass = "MemoryStoreDataManager", targetMethod = "<init>", targetLocation = "ENTRY", action = "createRendezvous($0, 2, true)"), @BMRule(name = "getGroupsContaining call", targetClass = "MemoryStoreDataManager", targetMethod = "getGroupsContaining", targetLocation = "ENTRY", action = "rendezvous($0); debug(Thread.currentThread().getName() + \": thread proceeding.\")") })
@Test
public void deadlockOnGroupContains() throws IndyDataException, InterruptedException, ExecutionException {
    ExecutorService executor = Executors.newFixedThreadPool(2);
    ExecutorCompletionService<String> completionService = new ExecutorCompletionService<>(executor);
    AtomicInteger count = new AtomicInteger(0);
    RemoteRepository repo = new RemoteRepository(MAVEN_PKG_KEY, "central", "http://repo.maven.apache.org/maven2");
    TestUpdatingEventDispatcher dispatcher = new TestUpdatingEventDispatcher(repo, completionService, count);
    MemoryStoreDataManager dataManager = new MemoryStoreDataManager(dispatcher, new DefaultIndyConfiguration());
    dispatcher.setDataManager(dataManager);
    ChangeSummary summary = new ChangeSummary(ChangeSummary.SYSTEM_USER, "Test init");
    dataManager.storeArtifactStore(repo, summary, false, false, new EventMetadata());
    for (int i = 0; i < 2; i++) {
        Group group = new Group(MAVEN_PKG_KEY, "group" + i);
        if (i % 2 == 0) {
            group.addConstituent(repo);
        }
        dataManager.storeArtifactStore(group, summary, false, false, new EventMetadata());
    }
    for (int i = 0; i < count.get(); i++) {
        Future<String> future = completionService.take();
        assertThat(future.get(), nullValue());
    }
}
Also used : Group(org.commonjava.indy.model.core.Group) ExecutorCompletionService(java.util.concurrent.ExecutorCompletionService) RemoteRepository(org.commonjava.indy.model.core.RemoteRepository) EventMetadata(org.commonjava.maven.galley.event.EventMetadata) AtomicInteger(java.util.concurrent.atomic.AtomicInteger) ExecutorService(java.util.concurrent.ExecutorService) ChangeSummary(org.commonjava.indy.audit.ChangeSummary) DefaultIndyConfiguration(org.commonjava.indy.conf.DefaultIndyConfiguration) Test(org.junit.Test) BMRules(org.jboss.byteman.contrib.bmunit.BMRules)

Example 10 with EventMetadata

use of org.commonjava.maven.galley.event.EventMetadata in project indy by Commonjava.

the class StoreEnablementManager method onStoreError.

//    public void onStoreUpdate( @Observes ArtifactStorePostUpdateEvent event )
//    {
//        for ( ArtifactStore store : event )
//        {
//            if ( store.isDisabled() )
//            {
//                String toStr = store.getMetadata( DISABLE_TIMEOUT );
//                if ( isNotEmpty( toStr ) )
//                {
//                    int timeout = Integer.parseInt( toStr );
//                    try
//                    {
//                        setReEnablementTimeout( store.getKey(), timeout );
//                    }
//                    catch ( IndySchedulerException e )
//                    {
//                        Logger logger = LoggerFactory.getLogger( getClass() );
//                        logger.error( String.format( "Failed to schedule re-enablement of %s.", store.getKey() ), e );
//                    }
//                }
//            }
////            else
////            {
////                try
////                {
////                    cancelReEnablementTimeout( store.getKey() );
////                }
////                catch ( IndySchedulerException e )
////                {
////                    Logger logger = LoggerFactory.getLogger( getClass() );
////                    logger.error( String.format( "Failed to delete re-enablement job for %s.", store.getKey() ), e );
////                }
////            }
//        }
//    }
public void onStoreError(@Observes IndyStoreErrorEvent evt) {
    Logger logger = LoggerFactory.getLogger(getClass());
    StoreKey key = evt.getStoreKey();
    Throwable error = evt.getError();
    try {
        ArtifactStore store = storeDataManager.getArtifactStore(key);
        if (store.getDisableTimeout() <= TIMEOUT_NEVER_DISABLE) {
            logger.debug("Disable-timeout set to {}, will never disable the repo", store.getDisableTimeout());
            store.setDisabled(false);
        } else {
            store.setDisabled(true);
        }
        final ChangeSummary changeSummary = new ChangeSummary(ChangeSummary.SYSTEM_USER, String.format("Disabling %s due to error: %s\n\nStack Trace:\n  %s", key, error, StringUtils.join(error.getStackTrace(), "\n  ")));
        storeDataManager.storeArtifactStore(store, changeSummary, false, true, new EventMetadata());
        logger.warn("{} has been disabled due to store-level error: {}\n Will re-enable in {} seconds.", key, error, config.getStoreDisableTimeoutSeconds());
        // TODO: How is it this doesn't duplicate the event handler method onStoreUpdate()...we're updating the store just above here.
        setReEnablementTimeout(key);
    } catch (IndyDataException e) {
        logger.error(String.format("Failed to disable %s on error: %s", key, error), e);
    } catch (IndySchedulerException e) {
        logger.error(String.format("Failed to schedule re-enablement of %s for retry.", key), e);
    }
}
Also used : IndyDataException(org.commonjava.indy.data.IndyDataException) ArtifactStore(org.commonjava.indy.model.core.ArtifactStore) Logger(org.slf4j.Logger) ChangeSummary(org.commonjava.indy.audit.ChangeSummary) StoreKey(org.commonjava.indy.model.core.StoreKey) IndySchedulerException(org.commonjava.indy.core.expire.IndySchedulerException) EventMetadata(org.commonjava.maven.galley.event.EventMetadata)

Aggregations

EventMetadata (org.commonjava.maven.galley.event.EventMetadata)95 Transfer (org.commonjava.maven.galley.model.Transfer)38 ChangeSummary (org.commonjava.indy.audit.ChangeSummary)37 Test (org.junit.Test)37 IndyDataException (org.commonjava.indy.data.IndyDataException)27 RemoteRepository (org.commonjava.indy.model.core.RemoteRepository)27 ConcreteResource (org.commonjava.maven.galley.model.ConcreteResource)20 StoreKey (org.commonjava.indy.model.core.StoreKey)18 IndyWorkflowException (org.commonjava.indy.IndyWorkflowException)17 Group (org.commonjava.indy.model.core.Group)17 ArtifactStore (org.commonjava.indy.model.core.ArtifactStore)15 Logger (org.slf4j.Logger)15 IOException (java.io.IOException)13 Inject (javax.inject.Inject)11 ApiOperation (io.swagger.annotations.ApiOperation)10 ApiResponses (io.swagger.annotations.ApiResponses)10 InputStream (java.io.InputStream)10 ArrayList (java.util.ArrayList)10 Path (javax.ws.rs.Path)10 SimpleProjectVersionRef (org.commonjava.maven.atlas.ident.ref.SimpleProjectVersionRef)9