Example 86 with ConcreteResource
use of org.commonjava.maven.galley.model.ConcreteResource in project indy by Commonjava.
the class ContentControllerTest method detectHtml_HtmlDoctypeDeclaration.
@Test
public void detectHtml_HtmlDoctypeDeclaration() throws Exception {
final ConcreteResource res = new ConcreteResource(new SimpleLocation("test:uri"), "file.html");
final Transfer tx = fixture.getCache().getTransfer(res);
PrintWriter writer = null;
try {
writer = new PrintWriter(new OutputStreamWriter(tx.openOutputStream(TransferOperation.GENERATE)));
writer.print("<!DOCTYPE html PUBLIC \"-//W3C//DTD XHTML 1.0 Strict//EN\"\n" + "\"http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd\">");
writer.flush();
} finally {
IOUtils.closeQuietly(writer);
}
assertThat(content.isHtmlContent(tx), equalTo(true));
}
Also used :
ConcreteResource(org.commonjava.maven.galley.model.ConcreteResource)
Transfer(org.commonjava.maven.galley.model.Transfer)
SimpleLocation(org.commonjava.maven.galley.model.SimpleLocation)
OutputStreamWriter(java.io.OutputStreamWriter)
PrintWriter(java.io.PrintWriter)
Test(org.junit.Test)
Example 87 with ConcreteResource
use of org.commonjava.maven.galley.model.ConcreteResource in project indy by Commonjava.
the class ContentControllerTest method detectHtml_SingleHtmlElementLineWithPrecedingWhitespace.
@Test
public void detectHtml_SingleHtmlElementLineWithPrecedingWhitespace() throws Exception {
final ConcreteResource res = new ConcreteResource(new SimpleLocation("test:uri"), "file.html");
final Transfer tx = fixture.getCache().getTransfer(res);
PrintWriter writer = null;
try {
writer = new PrintWriter(new OutputStreamWriter(tx.openOutputStream(TransferOperation.GENERATE)));
writer.print(" <html>");
writer.flush();
} finally {
IOUtils.closeQuietly(writer);
}
assertThat(content.isHtmlContent(tx), equalTo(true));
}
Also used :
ConcreteResource(org.commonjava.maven.galley.model.ConcreteResource)
Transfer(org.commonjava.maven.galley.model.Transfer)
SimpleLocation(org.commonjava.maven.galley.model.SimpleLocation)
OutputStreamWriter(java.io.OutputStreamWriter)
PrintWriter(java.io.PrintWriter)
Test(org.junit.Test)
Example 88 with ConcreteResource
use of org.commonjava.maven.galley.model.ConcreteResource in project indy by Commonjava.
the class PromotionHelper method clearStoreNFC.
/**
* Clear NFC for the source store and affected groups.
*
* @param sourcePaths The set of paths that need to be cleared from the NFC.
* @param store The store whose affected groups should have their NFC entries cleared
*/
public void clearStoreNFC(final Set<String> sourcePaths, final ArtifactStore store, final Set<Group> affectedGroups) {
Set<String> paths = sourcePaths.stream().map(sp -> sp.startsWith("/") && sp.length() > 1 ? sp.substring(1) : sp).collect(Collectors.toSet());
paths.forEach(path -> {
ConcreteResource resource = new ConcreteResource(LocationUtils.toLocation(store), path);
logger.trace("Clearing NFC path: {} from: {}\n\tResource: {}", path, store.getKey(), resource);
nfc.clearMissing(resource);
});
Set<Group> groups;
if (affectedGroups != null) {
groups = affectedGroups;
} else {
try {
groups = storeManager.query().getGroupsAffectedBy(store.getKey());
} catch (IndyDataException e) {
logger.warn("Failed to clear NFC for groups affected by " + store.getKey(), e);
return;
}
}
if (groups != null) {
groups.forEach(group -> {
KeyedLocation gl = LocationUtils.toLocation(group);
paths.forEach(path -> {
ConcreteResource resource = new ConcreteResource(gl, path);
logger.trace("Clearing NFC path: {} from: {}\n\tResource: {}", path, group.getKey(), resource);
nfc.clearMissing(resource);
});
});
}
}
Also used :
KeyedLocation(org.commonjava.indy.model.galley.KeyedLocation)
LoggerFactory(org.slf4j.LoggerFactory)
PathGauges(org.commonjava.indy.promote.metrics.PathGauges)
Group(org.commonjava.indy.model.core.Group)
ArrayList(java.util.ArrayList)
Inject(javax.inject.Inject)
Transfer(org.commonjava.maven.galley.model.Transfer)
IndyWorkflowException(org.commonjava.indy.IndyWorkflowException)
DownloadManager(org.commonjava.indy.content.DownloadManager)
StoreKey(org.commonjava.indy.model.core.StoreKey)
Location(org.commonjava.maven.galley.model.Location)
PathsPromoteRequest(org.commonjava.indy.promote.model.PathsPromoteRequest)
PromotionValidationException(org.commonjava.indy.promote.validate.PromotionValidationException)
LocationUtils(org.commonjava.indy.util.LocationUtils)
ConcreteResource(org.commonjava.maven.galley.model.ConcreteResource)
Logger(org.slf4j.Logger)
ArtifactStore(org.commonjava.indy.model.core.ArtifactStore)
StoreType(org.commonjava.indy.model.core.StoreType)
Set(java.util.Set)
PathsPromoteResult(org.commonjava.indy.promote.model.PathsPromoteResult)
NotFoundCache(org.commonjava.maven.galley.spi.nfc.NotFoundCache)
Collectors(java.util.stream.Collectors)
ContentManager(org.commonjava.indy.content.ContentManager)
TimeUnit(java.util.concurrent.TimeUnit)
List(java.util.List)
IGNORE_READONLY(org.commonjava.indy.data.StoreDataManager.IGNORE_READONLY)
EventMetadata(org.commonjava.maven.galley.event.EventMetadata)
ApplicationScoped(javax.enterprise.context.ApplicationScoped)
IndyDataException(org.commonjava.indy.data.IndyDataException)
StoreDataManager(org.commonjava.indy.data.StoreDataManager)
IndyDataException(org.commonjava.indy.data.IndyDataException)
Group(org.commonjava.indy.model.core.Group)
KeyedLocation(org.commonjava.indy.model.galley.KeyedLocation)
ConcreteResource(org.commonjava.maven.galley.model.ConcreteResource)
Example 89 with ConcreteResource
use of org.commonjava.maven.galley.model.ConcreteResource in project indy by Commonjava.
the class NPMContentAccessHandler method generateNPMContentsFromTransfer.
private List<Transfer> generateNPMContentsFromTransfer(final Transfer transfer, final EventMetadata eventMetadata) {
if (transfer == null || !transfer.exists()) {
return null;
}
Transfer versionTarget = null;
Transfer tarballTarget = null;
String versionContent = "";
String tarballContent = "";
ConcreteResource resource = transfer.getResource();
try {
ObjectMapper mapper = new ObjectMapper();
JsonNode root = mapper.readTree(transfer.openInputStream(true));
String versionPath = null;
String tarballPath = null;
JsonNode vnode = root.path("versions");
JsonNode anode = root.path("_attachments");
JsonNode idnode = root.path("_id");
if (vnode.fields().hasNext()) {
Map.Entry<String, JsonNode> entry = vnode.fields().next();
String version = entry.getKey();
if (version == null) {
return null;
}
versionPath = Paths.get(idnode.asText(), version).toString();
versionContent = entry.getValue().toString();
}
if (anode.fields().hasNext()) {
String tarball = anode.fields().next().getKey();
if (tarball == null) {
return null;
}
// remove scope if present
if (tarball.startsWith("@") && tarball.contains("/")) {
tarball = tarball.split("/", 2)[1];
}
tarballPath = Paths.get(idnode.asText(), "-", tarball).toString();
tarballContent = anode.findPath("data").asText();
}
if (versionPath == null || tarballPath == null) {
return null;
}
versionTarget = transfers.getCacheReference(new ConcreteResource(resource.getLocation(), versionPath));
tarballTarget = transfers.getCacheReference(new ConcreteResource(resource.getLocation(), tarballPath));
} catch (final IOException e) {
logger.error(String.format("[NPM] Json node parse failed for resource: %s. Reason: %s", resource, e.getMessage()), e);
}
if (versionTarget == null || tarballTarget == null) {
return null;
}
try (OutputStream versionOutputStream = versionTarget.openOutputStream(TransferOperation.UPLOAD, true, eventMetadata);
OutputStream tarballOutputStream = tarballTarget.openOutputStream(TransferOperation.UPLOAD, true, eventMetadata)) {
logger.info("STORE {}", versionTarget.getResource());
versionOutputStream.write(versionContent.getBytes());
logger.info("STORE {}", tarballTarget.getResource());
tarballOutputStream.write(Base64.decodeBase64(tarballContent));
return generateTransfers(versionTarget, tarballTarget);
} catch (final IOException e) {
logger.error(String.format("[NPM] Failed to store the generated targets: %s and %s. Reason: %s", versionTarget.getResource(), tarballTarget.getResource(), e.getMessage()), e);
}
return null;
}
Also used :
OutputStream(java.io.OutputStream)
Transfer(org.commonjava.maven.galley.model.Transfer)
ConcreteResource(org.commonjava.maven.galley.model.ConcreteResource)
JsonNode(com.fasterxml.jackson.databind.JsonNode)
IOException(java.io.IOException)
Map(java.util.Map)
ObjectMapper(com.fasterxml.jackson.databind.ObjectMapper)
Example 90 with ConcreteResource
use of org.commonjava.maven.galley.model.ConcreteResource in project indy by Commonjava.
the class MaintenanceController method getTombstoneStores.
public Set<StoreKey> getTombstoneStores(String packageType) throws IndyDataException {
List<HostedRepository> stores = storeDataManager.query().getAllHostedRepositories(packageType);
Set<StoreKey> tombstoneStores = new HashSet<>();
for (HostedRepository hosted : stores) {
StoreKey key = hosted.getKey();
ConcreteResource root = new ConcreteResource(LocationUtils.toLocation(hosted), PathUtils.ROOT);
String[] files = cacheProvider.list(root);
if (files == null || files.length == 0) {
logger.debug("Empty store: {}", key);
Set<Group> affected = storeDataManager.affectedBy(Arrays.asList(key));
if (affected == null || affected.isEmpty()) {
logger.info("Find tombstone store (no content and not in any group): {}", key);
tombstoneStores.add(key);
}
}
}
return tombstoneStores;
}
Also used :
Group(org.commonjava.indy.model.core.Group)
ConcreteResource(org.commonjava.maven.galley.model.ConcreteResource)
StoreKey(org.commonjava.indy.model.core.StoreKey)
HostedRepository(org.commonjava.indy.model.core.HostedRepository)
HashSet(java.util.HashSet)
Aggregations
ConcreteResource (org.commonjava.maven.galley.model.ConcreteResource)206
Test (org.junit.Test)138
Transfer (org.commonjava.maven.galley.model.Transfer)92
SimpleLocation (org.commonjava.maven.galley.model.SimpleLocation)55
Location (org.commonjava.maven.galley.model.Location)45
EventMetadata (org.commonjava.maven.galley.event.EventMetadata)36
BMScript (org.jboss.byteman.contrib.bmunit.BMScript)33
OutputStream (java.io.OutputStream)30
InputStream (java.io.InputStream)26
Group (org.commonjava.indy.model.core.Group)25
ArrayList (java.util.ArrayList)23
HostedRepository (org.commonjava.indy.model.core.HostedRepository)23
TransferException (org.commonjava.maven.galley.TransferException)23
SimpleHttpLocation (org.commonjava.maven.galley.transport.htcli.model.SimpleHttpLocation)23
HashSet (java.util.HashSet)22
IndyWorkflowException (org.commonjava.indy.IndyWorkflowException)18
KeyedLocation (org.commonjava.indy.model.galley.KeyedLocation)18
Logger (org.slf4j.Logger)18
ByteArrayInputStream (java.io.ByteArrayInputStream)17
RemoteRepository (org.commonjava.indy.model.core.RemoteRepository)16
