Search in sources :

Example 1 with SolrServerFacade

use of cz.mzk.recordmanager.server.solr.SolrServerFacade in project RecordManager2 by moravianlibrary.

the class DeleteAllRecordsFromSolrJobConfig method deleteTasklet.

@Bean(name = "deleteAllRecordsFromSolrJob:deleteTasklet")
@StepScope
public Tasklet deleteTasklet(@Value("#{jobParameters[" + Constants.JOB_PARAM_SOLR_URL + "]}") final String solrUrl, @Value("#{jobParameters[" + Constants.JOB_PARAM_SOLR_QUERY + "]}") final String solrQuery) {
    final String query = (solrQuery != null) ? solrQuery : ALL_DOCUMENTS_QUERY;
    return new Tasklet() {

        @Override
        public RepeatStatus execute(StepContribution contribution, ChunkContext chunkContext) throws Exception {
            SolrServerFacade server = factory.create(solrUrl);
            server.deleteByQuery(query);
            server.commit();
            return RepeatStatus.FINISHED;
        }
    };
}
Also used : StepContribution(org.springframework.batch.core.StepContribution) Tasklet(org.springframework.batch.core.step.tasklet.Tasklet) ChunkContext(org.springframework.batch.core.scope.context.ChunkContext) SolrServerFacade(cz.mzk.recordmanager.server.solr.SolrServerFacade) StepScope(org.springframework.batch.core.configuration.annotation.StepScope) Bean(org.springframework.context.annotation.Bean)

Example 2 with SolrServerFacade

use of cz.mzk.recordmanager.server.solr.SolrServerFacade in project RecordManager2 by moravianlibrary.

the class IndexIndividualRecordsToSolrJobTest method execute.

@Test
public void execute() throws Exception {
    SolrServerFacade server = solrServerFactory.create(SOLR_URL);
    Job job = jobRegistry.getJob("indexIndividualRecordsToSolrJob");
    Map<String, JobParameter> params = new HashMap<String, JobParameter>();
    params.put(Constants.JOB_PARAM_SOLR_URL, new JobParameter(SOLR_URL));
    params.put(Constants.JOB_PARAM_RECORD_IDS, new JobParameter("MZK.MZK01-000000117"));
    JobParameters jobParams = new JobParameters(params);
    JobExecution execution = jobLauncher.run(job, jobParams);
    Assert.assertEquals(execution.getExitStatus(), ExitStatus.COMPLETED);
    {
        SolrQuery allDocsQuery = new SolrQuery();
        allDocsQuery.set("q", "id:60");
        allDocsQuery.set("rows", 10000);
        QueryResponse allDocsResponse = server.query(allDocsQuery);
        Assert.assertEquals(allDocsResponse.getResults().size(), 1);
    }
}
Also used : JobExecution(org.springframework.batch.core.JobExecution) HashMap(java.util.HashMap) QueryResponse(org.apache.solr.client.solrj.response.QueryResponse) JobParameters(org.springframework.batch.core.JobParameters) Job(org.springframework.batch.core.Job) JobParameter(org.springframework.batch.core.JobParameter) SolrQuery(org.apache.solr.client.solrj.SolrQuery) SolrServerFacade(cz.mzk.recordmanager.server.solr.SolrServerFacade) Test(org.testng.annotations.Test)

Example 3 with SolrServerFacade

use of cz.mzk.recordmanager.server.solr.SolrServerFacade in project RecordManager2 by moravianlibrary.

the class IndexAllRecordsToEmbeddedSolrJobTest method execute.

@Test
public void execute() throws Exception {
    SolrServerFacade server = solrServerFactory.create(SOLR_URL);
    Job job = jobRegistry.getJob(Constants.JOB_ID_SOLR_INDEX_ALL_RECORDS);
    Map<String, JobParameter> params = new HashMap<String, JobParameter>();
    params.put(Constants.JOB_PARAM_SOLR_URL, new JobParameter(SOLR_URL));
    JobParameters jobParams = new JobParameters(params);
    JobExecution execution = jobLauncher.run(job, jobParams);
    Assert.assertEquals(execution.getExitStatus(), ExitStatus.COMPLETED);
    SolrQuery query = new SolrQuery();
    query.set("q", "id:*");
    query.setRows(100);
    QueryResponse response = server.query(query);
    Assert.assertEquals(response.getResults().size(), 85);
}
Also used : JobExecution(org.springframework.batch.core.JobExecution) HashMap(java.util.HashMap) QueryResponse(org.apache.solr.client.solrj.response.QueryResponse) JobParameters(org.springframework.batch.core.JobParameters) Job(org.springframework.batch.core.Job) JobParameter(org.springframework.batch.core.JobParameter) SolrQuery(org.apache.solr.client.solrj.SolrQuery) SolrServerFacade(cz.mzk.recordmanager.server.solr.SolrServerFacade) Test(org.testng.annotations.Test)

Example 4 with SolrServerFacade

use of cz.mzk.recordmanager.server.solr.SolrServerFacade in project RecordManager2 by moravianlibrary.

the class IndexIndividualHarvestedRecordsToSolrJobTest method execute.

@Test
public void execute() throws Exception {
    SolrServerFacade server = solrServerFactory.create(SOLR_URL);
    Job job = jobRegistry.getJob("indexIndividualHarvestedRecordsToSolrJob");
    Map<String, JobParameter> params = new HashMap<String, JobParameter>();
    params.put(Constants.JOB_PARAM_SOLR_URL, new JobParameter(SOLR_URL));
    params.put(Constants.JOB_PARAM_RECORD_IDS, new JobParameter("MZK01-000000117"));
    JobParameters jobParams = new JobParameters(params);
    JobExecution execution = jobLauncher.run(job, jobParams);
    Assert.assertEquals(execution.getExitStatus(), ExitStatus.COMPLETED);
    {
        SolrQuery allDocsQuery = new SolrQuery();
        allDocsQuery.set("q", "id:MZK.MZK01-000000117");
        allDocsQuery.set("rows", 10000);
        QueryResponse allDocsResponse = server.query(allDocsQuery);
        Assert.assertEquals(allDocsResponse.getResults().size(), 1);
    }
}
Also used : JobExecution(org.springframework.batch.core.JobExecution) HashMap(java.util.HashMap) QueryResponse(org.apache.solr.client.solrj.response.QueryResponse) JobParameters(org.springframework.batch.core.JobParameters) Job(org.springframework.batch.core.Job) JobParameter(org.springframework.batch.core.JobParameter) SolrQuery(org.apache.solr.client.solrj.SolrQuery) SolrServerFacade(cz.mzk.recordmanager.server.solr.SolrServerFacade) Test(org.testng.annotations.Test)

Example 5 with SolrServerFacade

use of cz.mzk.recordmanager.server.solr.SolrServerFacade in project RecordManager2 by moravianlibrary.

the class IndexRecordsToEmbeddedSolrJobTest method execute.

@Test
public void execute() throws Exception {
    SolrServerFacade server = solrServerFactory.create(SOLR_URL);
    Job job = jobRegistry.getJob("indexAllRecordsToSolrJob");
    Map<String, JobParameter> params = new HashMap<String, JobParameter>();
    params.put(Constants.JOB_PARAM_SOLR_URL, new JobParameter(SOLR_URL));
    JobParameters jobParams = new JobParameters(params);
    JobExecution execution = jobLauncher.run(job, jobParams);
    Assert.assertEquals(execution.getExitStatus(), ExitStatus.COMPLETED);
    {
        SolrQuery allDocsQuery = new SolrQuery();
        allDocsQuery.set("q", SolrFieldConstants.ID_FIELD + ":*");
        allDocsQuery.set("rows", 10000);
        QueryResponse allDocsResponse = server.query(allDocsQuery);
        Assert.assertEquals(allDocsResponse.getResults().size(), 85);
    }
    {
        SolrQuery docQuery = new SolrQuery();
        docQuery.set("q", SolrFieldConstants.ID_FIELD + ":64");
        QueryResponse docResponse = server.query(docQuery);
        Assert.assertEquals(docResponse.getResults().size(), 1);
        SolrDocument document = docResponse.getResults().get(0);
        Assert.assertEquals(document.get(SolrFieldConstants.AUTHOR_FIELD), "Grisham, John, 1955-");
        Assert.assertEquals(document.get(SolrFieldConstants.TITLE), "--a je čas zabíjet /");
    }
    {
        SolrQuery docQuery = new SolrQuery();
        docQuery.set("q", SolrFieldConstants.ID_FIELD + ":91");
        QueryResponse docResponse = server.query(docQuery);
        Assert.assertEquals(docResponse.getResults().size(), 1);
        SolrDocument document = docResponse.getResults().get(0);
        Assert.assertEquals(document.get(SolrFieldConstants.AUTHOR_FIELD), "Andrić, Ivo, 1892-1975");
        Assert.assertEquals(document.get(SolrFieldConstants.TITLE), "Most přes Drinu");
    }
    {
        SolrQuery deletedDocQuery = new SolrQuery();
        deletedDocQuery.set("q", SolrFieldConstants.ID_FIELD + ":99");
        QueryResponse docResponse = server.query(deletedDocQuery);
        Assert.assertEquals(docResponse.getResults().size(), 0);
    }
    {
        SolrQuery authQuery = new SolrQuery();
        authQuery.set("q", SolrFieldConstants.ID_FIELD + ":73");
        QueryResponse docResponse = server.query(authQuery);
        Assert.assertEquals(docResponse.getResults().size(), 1);
        SolrDocument document = docResponse.getResults().get(0);
        // check whether author_search field contains alternative name from authority record
        Assert.assertTrue(// 
        document.getFieldValues(SolrFieldConstants.AUTHOR_VIZ_FIELD).stream().anyMatch(s -> s.equals("Imaginarni, Karel, 1900-2000")), "Authority enrichment failed.");
    }
    {
        SolrQuery authQuery = new SolrQuery();
        authQuery.set("q", SolrFieldConstants.ID_FIELD + ":103");
        QueryResponse docResponse = server.query(authQuery);
        Assert.assertEquals(docResponse.getResults().size(), 1);
        SolrDocument document = docResponse.getResults().get(0);
        // check whether subject_viz field contains alternative name from mesh
        Assert.assertTrue(// 
        document.getFieldValues(SolrFieldConstants.SUBJECT_VIZ_FIELD).stream().anyMatch(s -> s.equals("klinická onkologie")), "Mesh enrichment failed.");
    }
    {
        SolrQuery dcQuery = new SolrQuery();
        dcQuery.set("q", SolrFieldConstants.ID_FIELD + ":100");
        QueryResponse docResponse = server.query(dcQuery);
        Assert.assertEquals(docResponse.getResults().size(), 1);
        SolrDocument document = docResponse.getResults().get(0);
        Assert.assertTrue(document.containsKey(SolrFieldConstants.RECORDTYPE), "Record type missing in DC record.");
        Assert.assertEquals(document.getFieldValue(SolrFieldConstants.RECORDTYPE), "dublinCore");
        Assert.assertTrue(document.containsKey(SolrFieldConstants.FULLTEXT_FIELD), "Full record missing in DC record.");
        String fullRecord = (String) document.getFieldValue(SolrFieldConstants.FULLTEXT_FIELD);
        Assert.assertTrue(fullRecord.length() > 0);
    }
    {
        // check for url
        SolrQuery dcQuery = new SolrQuery();
        dcQuery.set("q", SolrFieldConstants.ID_FIELD + ":96");
        QueryResponse docResponse = server.query(dcQuery);
        Assert.assertEquals(docResponse.getResults().size(), 1);
        SolrDocument document = docResponse.getResults().get(0);
        Assert.assertTrue(document.getFieldValues(SolrFieldConstants.URL).stream().anyMatch(url -> url.equals("MZK|unknown|http://krameriusndktest.mzk.cz/search/handle/uuid:f1401080-de25-11e2-9923-005056827e52|Digitalizovaný dokument")));
    }
    {
        // check for Kramerius url
        SolrQuery dcQuery = new SolrQuery();
        dcQuery.set("q", SolrFieldConstants.ID_FIELD + ":100");
        QueryResponse docResponse = server.query(dcQuery);
        Assert.assertEquals(docResponse.getResults().size(), 1);
        SolrDocument document = docResponse.getResults().get(0);
        Assert.assertTrue(document.getFieldValues(SolrFieldConstants.URL).stream().anyMatch(url -> url.equals("kram-mzk|online|http://www.digitalniknihovna.cz/mzk/uuid/UUID:039764f8-d6db-11e0-b2cd-0050569d679d|")));
    }
    {
        // check for indexed fulltext
        SolrQuery dcQuery = new SolrQuery();
        dcQuery.set("q", SolrFieldConstants.ID_FIELD + ":100 AND " + SolrFieldConstants.FULLTEXT_FIELD + ":indexace");
        QueryResponse docResponse = server.query(dcQuery);
        Assert.assertEquals(docResponse.getResults().size(), 1);
    }
    {
        // check for indexed fulltext
        SolrQuery dcQuery = new SolrQuery();
        dcQuery.set("q", SolrFieldConstants.ID_FIELD + ":100 AND " + SolrFieldConstants.FULLTEXT_FIELD + ":neexistuje");
        QueryResponse docResponse = server.query(dcQuery);
        Assert.assertEquals(docResponse.getResults().size(), 0);
    }
    {
        // check dedup record for authority record
        SolrQuery docQuery = new SolrQuery();
        docQuery.set("q", SolrFieldConstants.ID_FIELD + ":102");
        QueryResponse docResponse = server.query(docQuery);
        Assert.assertEquals(docResponse.getResults().size(), 1);
        SolrDocument document = docResponse.getResults().get(0);
        Assert.assertEquals(document.get(SolrFieldConstants.ID_AUTHORITY), "aut000001");
        Assert.assertFalse(document.containsKey(SolrFieldConstants.AUTHOR_FIELD));
    }
    {
        // check authority record
        SolrQuery docQuery = new SolrQuery();
        docQuery.set("q", SolrFieldConstants.ID_FIELD + ":auth.AUT10-000051020");
        QueryResponse docResponse = server.query(docQuery);
        Assert.assertEquals(docResponse.getResults().size(), 1);
        SolrDocument document = docResponse.getResults().get(0);
        Assert.assertEquals(document.get(SolrFieldConstants.USE_FOR), Collections.singletonList("Karel Imaginarni, 1900-2000"));
        Assert.assertEquals(document.get(SolrFieldConstants.HEADING), "Jiří Šolc, 1900-2000");
    }
    {
        // check sfx record
        SolrQuery docQuery = new SolrQuery();
        docQuery.set("q", SolrFieldConstants.ID_FIELD + ":104");
        QueryResponse docResponse = server.query(docQuery);
        Assert.assertEquals(docResponse.getResults().size(), 1);
        SolrDocument document = docResponse.getResults().get(0);
        Assert.assertEquals(document.get(SolrFieldConstants.LOCAL_IDS_FIELD), Collections.singletonList("sfxjibmzk.1000000000000004"));
        Assert.assertEquals(document.get(SolrFieldConstants.SFX_SOURCE_TXT), "mzk");
    }
}
Also used : JobExecution(org.springframework.batch.core.JobExecution) SolrDocument(org.apache.solr.common.SolrDocument) HashMap(java.util.HashMap) QueryResponse(org.apache.solr.client.solrj.response.QueryResponse) JobParameters(org.springframework.batch.core.JobParameters) Job(org.springframework.batch.core.Job) JobParameter(org.springframework.batch.core.JobParameter) SolrQuery(org.apache.solr.client.solrj.SolrQuery) SolrServerFacade(cz.mzk.recordmanager.server.solr.SolrServerFacade) Test(org.testng.annotations.Test)

Aggregations

SolrServerFacade (cz.mzk.recordmanager.server.solr.SolrServerFacade)9 SolrQuery (org.apache.solr.client.solrj.SolrQuery)6 QueryResponse (org.apache.solr.client.solrj.response.QueryResponse)6 HashMap (java.util.HashMap)4 Job (org.springframework.batch.core.Job)4 JobExecution (org.springframework.batch.core.JobExecution)4 JobParameter (org.springframework.batch.core.JobParameter)4 JobParameters (org.springframework.batch.core.JobParameters)4 Test (org.testng.annotations.Test)4 HarvestedRecord (cz.mzk.recordmanager.server.model.HarvestedRecord)2 SolrServerException (org.apache.solr.client.solrj.SolrServerException)2 SolrDocumentList (org.apache.solr.common.SolrDocumentList)2 SolrInputDocument (org.apache.solr.common.SolrInputDocument)2 DedupRecord (cz.mzk.recordmanager.server.model.DedupRecord)1 ArrayList (java.util.ArrayList)1 SolrDocument (org.apache.solr.common.SolrDocument)1 StepContribution (org.springframework.batch.core.StepContribution)1 StepScope (org.springframework.batch.core.configuration.annotation.StepScope)1 ChunkContext (org.springframework.batch.core.scope.context.ChunkContext)1 Tasklet (org.springframework.batch.core.step.tasklet.Tasklet)1