Search in sources :

Example 1 with Part

use of net.codestory.http.Part in project datashare by ICIJ.

the class BatchSearchResource method search.

/**
 * Creates a new batch search. This is a multipart form with 8 fields :
 * name, description, csvFile, published, fileTypes, paths, fuzziness, phrase_matches
 *
 * No matter the order. The name and csv file are mandatory else it will return 400 (bad request)
 * Csv file must have under 60 000 lines else it will return 413 (payload too large)
 * Queries with less than two characters are filtered
 *
 * To do so with bash you can create a text file like :
 * ```
 * --BOUNDARY
 * Content-Disposition: form-data; name="name"
 *
 * my batch search
 * --BOUNDARY
 * Content-Disposition: form-data; name="description"
 *
 * search description
 * --BOUNDARY
 * Content-Disposition: form-data; name="csvFile"; filename="search.csv"
 * Content-Type: text/csv
 *
 * Obama
 * skype
 * test
 * query three
 * --BOUNDARY--
 * Content-Disposition: form-data; name="published"
 *
 * true
 * --BOUNDARY--
 * ```
 * Then replace `\n` with `\r\n` with a sed like this:
 *
 * `sed -i 's/$/^M/g' ~/multipart.txt`
 *
 * Then make a curl request with this file :
 * ```
 * curl -i -XPOST localhost:8080/api/batch/search/prj -H 'Content-Type: multipart/form-data; boundary=BOUNDARY' --data-binary @/home/dev/multipart.txt
 * ```
 * @param projectId
 * @param context : the request body
 * @return 200 or 400 or 413
 */
@Post("/search/:project")
public Payload search(String projectId, Context context) throws Exception {
    List<Part> parts = context.parts();
    String name = fieldValue("name", parts);
    String csv = fieldValue("csvFile", parts);
    if (name == null || csv == null) {
        return badRequest();
    }
    String description = fieldValue("description", parts);
    boolean published = "true".equalsIgnoreCase(fieldValue("published", parts)) ? TRUE : FALSE;
    List<String> fileTypes = fieldValues("fileTypes", parts);
    List<String> paths = fieldValues("paths", parts);
    Optional<Part> fuzzinessPart = parts.stream().filter(p -> "fuzziness".equals(p.name())).findAny();
    int fuzziness = fuzzinessPart.isPresent() ? parseInt(fuzzinessPart.get().content()) : 0;
    Optional<Part> phraseMatchesPart = parts.stream().filter(p -> "phrase_matches".equals(p.name())).findAny();
    boolean phraseMatches = phraseMatchesPart.isPresent() ? parseBoolean(phraseMatchesPart.get().content()) : FALSE;
    LinkedHashSet<String> queries = getQueries(csv).stream().map(query -> (phraseMatches && query.contains("\"")) ? query : sanitizeDoubleQuotesInQuery(query)).collect(Collectors.toCollection(LinkedHashSet::new));
    if (queries.size() >= MAX_BATCH_SIZE)
        return new Payload(413);
    BatchSearch batchSearch = new BatchSearch(project(projectId), name, description, queries, (User) context.currentUser(), published, fileTypes, paths, fuzziness, phraseMatches);
    boolean isSaved = batchSearchRepository.save(batchSearch);
    if (isSaved)
        batchSearchQueue.put(batchSearch.uuid);
    return isSaved ? new Payload("application/json", batchSearch.uuid, 200) : badRequest();
}
Also used : java.util(java.util) Inject(com.google.inject.Inject) UnauthorizedException(net.codestory.http.errors.UnauthorizedException) SearchResult(org.icij.datashare.batch.SearchResult) JooqBatchSearchRepository(org.icij.datashare.db.JooqBatchSearchRepository) Boolean(java.lang.Boolean) User(org.icij.datashare.user.User) Project.project(org.icij.datashare.text.Project.project) DatashareUser(org.icij.datashare.session.DatashareUser) Part(net.codestory.http.Part) net.codestory.http.annotations(net.codestory.http.annotations) PropertiesProvider(org.icij.datashare.PropertiesProvider) Project(org.icij.datashare.text.Project) Context(net.codestory.http.Context) Payload(net.codestory.http.payload.Payload) CollectionUtils.asSet(org.icij.datashare.CollectionUtils.asSet) IOException(java.io.IOException) BlockingQueue(java.util.concurrent.BlockingQueue) BatchSearchRecord(org.icij.datashare.batch.BatchSearchRecord) NotFoundException(net.codestory.http.errors.NotFoundException) BatchSearchRepository(org.icij.datashare.batch.BatchSearchRepository) Collectors(java.util.stream.Collectors) Integer.parseInt(java.lang.Integer.parseInt) String.format(java.lang.String.format) BatchSearch(org.icij.datashare.batch.BatchSearch) Arrays.stream(java.util.Arrays.stream) Singleton(com.google.inject.Singleton) BatchSearch(org.icij.datashare.batch.BatchSearch) Part(net.codestory.http.Part) Payload(net.codestory.http.payload.Payload)

Aggregations

Inject (com.google.inject.Inject)1 Singleton (com.google.inject.Singleton)1 IOException (java.io.IOException)1 Boolean (java.lang.Boolean)1 Integer.parseInt (java.lang.Integer.parseInt)1 String.format (java.lang.String.format)1 java.util (java.util)1 Arrays.stream (java.util.Arrays.stream)1 BlockingQueue (java.util.concurrent.BlockingQueue)1 Collectors (java.util.stream.Collectors)1 Context (net.codestory.http.Context)1 Part (net.codestory.http.Part)1 net.codestory.http.annotations (net.codestory.http.annotations)1 NotFoundException (net.codestory.http.errors.NotFoundException)1 UnauthorizedException (net.codestory.http.errors.UnauthorizedException)1 Payload (net.codestory.http.payload.Payload)1 CollectionUtils.asSet (org.icij.datashare.CollectionUtils.asSet)1 PropertiesProvider (org.icij.datashare.PropertiesProvider)1 BatchSearch (org.icij.datashare.batch.BatchSearch)1 BatchSearchRecord (org.icij.datashare.batch.BatchSearchRecord)1