Search in sources :

Example 1 with PacketHandleException

use of me.retrodaredevil.solarthing.packets.handling.PacketHandleException in project solarthing by wildmountainfarms.

the class CouchDbPacketSaver method handle.

@Override
public void handle(PacketCollection packetCollection) throws PacketHandleException {
    // Normally we would try and create the database, but that doesn't work with non-admin cookie authenticated users
    String id = packetCollection.getDbId();
    String revision = idMap == null ? null : idMap.get(id);
    final JsonData jsonData;
    try {
        jsonData = new StringJsonData(MAPPER.writeValueAsString(packetCollection));
    } catch (JsonProcessingException e) {
        throw new RuntimeException("Cannot serialize packet collection! This is bad!", e);
    }
    try {
        final DocumentResponse response;
        if (revision == null) {
            response = database.putDocument(id, jsonData);
        } else {
            response = database.updateDocument(id, revision, jsonData);
        }
        LOGGER.debug("Now revision is: " + response.getRev() + ". It was: " + revision);
        if (idMap != null) {
            // Currently, if we have a new document ID, we never, ever, need to worry about using an older document ID, so we can clear the map to avoid keeping unnecessary memory
            idMap.clear();
            idMap.put(id, response.getRev());
        }
    } catch (CouchDbNotFoundException ex) {
        throw new PacketHandleException("Got 'not found'. Does the database exist? Make sure to run the couchdb-setup!", ex);
    } catch (CouchDbUpdateConflictException ex) {
        if (idMap == null) {
            // we are ignoring conflicts
            LOGGER.debug("Got update conflict exception. Ignoring...");
            return;
        }
        try {
            String actualRev = database.getCurrentRevision(id);
            idMap.put(id, actualRev);
            LOGGER.debug("We were able to get the actual Revision ID for id=" + id + " actual rev=" + actualRev);
        } catch (CouchDbException revEx) {
            LOGGER.debug("Unable to get the actual Revision ID for id=" + id, revEx);
        }
        throw new PacketHandleException("Conflict while saving something to couchdb. id=" + id + " rev=" + revision + ". This usually means we put a packet in the database, but we weren't able to cache its rev id.", ex);
    } catch (CouchDbException ex) {
        if (ex.getCause() instanceof IOException) {
            throw new PacketHandleException("We got a DbAccessException probably meaning we couldn't reach the database.", ex);
        } else {
            throw new PacketHandleException("Got a DbAccessException without IOException as a cause. Something must be wrong.", ex);
        }
    }
}
Also used : StringJsonData(me.retrodaredevil.couchdbjava.json.StringJsonData) CouchDbException(me.retrodaredevil.couchdbjava.exception.CouchDbException) DocumentResponse(me.retrodaredevil.couchdbjava.response.DocumentResponse) CouchDbNotFoundException(me.retrodaredevil.couchdbjava.exception.CouchDbNotFoundException) IOException(java.io.IOException) CouchDbUpdateConflictException(me.retrodaredevil.couchdbjava.exception.CouchDbUpdateConflictException) JsonProcessingException(com.fasterxml.jackson.core.JsonProcessingException) JsonData(me.retrodaredevil.couchdbjava.json.JsonData) StringJsonData(me.retrodaredevil.couchdbjava.json.StringJsonData) PacketHandleException(me.retrodaredevil.solarthing.packets.handling.PacketHandleException)

Example 2 with PacketHandleException

use of me.retrodaredevil.solarthing.packets.handling.PacketHandleException in project solarthing by wildmountainfarms.

the class InfluxDbPacketSaver method handle.

@Override
public void handle(PacketCollection packetCollection) throws PacketHandleException {
    try (InfluxDB db = createDatabase()) {
        final InstancePacketGroup packetGroup = PacketGroups.parseToInstancePacketGroup(packetCollection, DefaultInstanceOptions.REQUIRE_NO_DEFAULTS);
        DefaultInstanceOptions.requireNoDefaults(packetGroup);
        final String database = databaseNameGetter.getName(packetGroup);
        try {
            QueryResult result = db.query(new Query("CREATE DATABASE " + database, null, true));
            String error = getError(result);
            if (error != null) {
                throw new PacketHandleException("Result got error! error: " + result);
            }
        } catch (InfluxDBException ex) {
            throw new PacketHandleException("Unable to query the database!", ex);
        }
        final RetentionPolicySetting retentionPolicySetting = retentionPolicyGetter.getRetentionPolicySetting();
        final String retentionPolicyName;
        // region Retention Policy Creation Logic
        if (retentionPolicySetting != null) {
            retentionPolicyName = retentionPolicySetting.getName();
            if (retentionPolicyName != null) {
                final RetentionPolicy policy = retentionPolicySetting.getRetentionPolicy();
                if (policy != null) {
                    final String policyString = policy.toPolicyStringInfluxDb1(retentionPolicyName, database);
                    final boolean needsAlter;
                    if (retentionPolicySetting.isTryToCreate()) {
                        final QueryResult result;
                        final String query = "CREATE " + policyString;
                        try {
                            result = db.query(new Query(query, null, true));
                        } catch (InfluxDBException ex) {
                            throw new PacketHandleException("Unable to query database to create retention policy: " + retentionPolicyName + " query: " + query, ex);
                        }
                        String error = getError(result);
                        if (retentionPolicySetting.isIgnoreUnsuccessfulCreate()) {
                            if (error != null) {
                                LOGGER.debug("We're going to ignore this error we got while trying to create a retention policy. Error: {}", error);
                            }
                            needsAlter = false;
                        } else {
                            if (error != null) {
                                LOGGER.debug("Got error while trying to create! Error: " + error);
                            }
                            needsAlter = error != null;
                        }
                        if (needsAlter && !retentionPolicySetting.isAutomaticallyAlter()) {
                            throw new PacketHandleException("Got error while trying to create retention policy: " + retentionPolicyName + ". Error: " + error);
                        }
                    } else {
                        needsAlter = true;
                    }
                    if (needsAlter) {
                        if (retentionPolicySetting.isAutomaticallyAlter()) {
                            final QueryResult alterResult;
                            try {
                                alterResult = db.query(new Query("ALTER " + policyString));
                                LOGGER.info(SolarThingConstants.SUMMARY_MARKER, "Successfully altered {} retention policy!", retentionPolicyName);
                            } catch (InfluxDBException ex) {
                                throw new PacketHandleException("Unable to query database to alter retention policy: " + retentionPolicyName, ex);
                            }
                            String error = getError(alterResult);
                            if (error != null) {
                                throw new PacketHandleException("Unable to alter retention policy: " + retentionPolicyName + ". Error: " + error);
                            }
                        } else {
                            throw new PacketHandleException("Retention policy: " + retentionPolicyName + " needs to be altered but automatically alter is false!");
                        }
                    }
                }
            }
        } else {
            retentionPolicyName = null;
        }
        // endregion
        final long time = packetCollection.getDateMillis();
        final BatchPoints points = BatchPoints.database(database).tag("sourceId", packetGroup.getSourceId()).tag("fragmentId", "" + packetGroup.getFragmentId()).consistency(InfluxDB.ConsistencyLevel.ALL).retentionPolicy(// may be null, but that's OK
        retentionPolicyName).build();
        int packetsWritten = 0;
        for (Packet packet : packetGroup.getPackets()) {
            Point.Builder pointBuilder = pointCreator.createBuilder(packet).time(time, TimeUnit.MILLISECONDS);
            Collection<String> tagKeys = PointUtil.getTagKeys(packet.getClass());
            ObjectNode json = OBJECT_MAPPER.valueToTree(packet);
            for (Map.Entry<String, ValueNode> entry : PointUtil.flattenJsonObject(json)) {
                String key = entry.getKey();
                ValueNode prim = entry.getValue();
                if (tagKeys.contains(key)) {
                    pointBuilder.tag(key, prim.asText());
                }
                if (prim.isNumber()) {
                    // always store as float datatype
                    pointBuilder.addField(key, prim.asDouble());
                } else if (prim.isTextual() || prim.isBinary()) {
                    pointBuilder.addField(key, prim.asText());
                } else if (prim.isBoolean()) {
                    pointBuilder.addField(key, prim.asBoolean());
                } else
                    throw new AssertionError("This primitive isn't a number, string/binary or boolean! It's: " + prim + " class: " + prim.getClass() + " text=" + prim.asText());
            }
            points.point(pointBuilder.build());
            packetsWritten++;
        }
        try {
            db.write(points);
        } catch (InfluxDBException ex) {
            throw new PacketHandleException("We were able to query the database, but unable to write the points to it!", ex);
        }
        LOGGER.debug("Wrote {} packets to InfluxDB! database={} retention policy={}", packetsWritten, database, retentionPolicyName);
    }
}
Also used : InstancePacketGroup(me.retrodaredevil.solarthing.packets.collection.InstancePacketGroup) Packet(me.retrodaredevil.solarthing.packets.Packet) Query(org.influxdb.dto.Query) BatchPoints(org.influxdb.dto.BatchPoints) ObjectNode(com.fasterxml.jackson.databind.node.ObjectNode) Point(org.influxdb.dto.Point) RetentionPolicy(me.retrodaredevil.solarthing.influxdb.retention.RetentionPolicy) Point(org.influxdb.dto.Point) InfluxDBException(org.influxdb.InfluxDBException) QueryResult(org.influxdb.dto.QueryResult) InfluxDB(org.influxdb.InfluxDB) RetentionPolicySetting(me.retrodaredevil.solarthing.influxdb.retention.RetentionPolicySetting) ValueNode(com.fasterxml.jackson.databind.node.ValueNode) Map(java.util.Map) PacketHandleException(me.retrodaredevil.solarthing.packets.handling.PacketHandleException)

Example 3 with PacketHandleException

use of me.retrodaredevil.solarthing.packets.handling.PacketHandleException in project solarthing by wildmountainfarms.

the class PostPacketHandler method handle.

@Override
public void handle(PacketCollection packetCollection) throws PacketHandleException {
    String string = stringPacketHandler.getString(packetCollection);
    Call call = client.newCall(new Request.Builder().url(url).post(RequestBody.create(string, mediaType)).build());
    try {
        Response response = call.execute();
        if (!response.isSuccessful()) {
            throw new PacketHandleException("Connected with unsuccessful response! code: " + response.code() + " message: " + response.message());
        }
    } catch (IOException e) {
        throw new PacketHandleException("Exception while posting!", e);
    }
}
Also used : IOException(java.io.IOException) PacketHandleException(me.retrodaredevil.solarthing.packets.handling.PacketHandleException)

Example 4 with PacketHandleException

use of me.retrodaredevil.solarthing.packets.handling.PacketHandleException in project solarthing by wildmountainfarms.

the class InfluxDb2PacketSaver method handle.

@Override
public void handle(PacketCollection packetCollection) throws PacketHandleException {
    final InstancePacketGroup packetGroup = PacketGroups.parseToInstancePacketGroup(packetCollection, DefaultInstanceOptions.REQUIRE_NO_DEFAULTS);
    DefaultInstanceOptions.requireNoDefaults(packetGroup);
    Organization organization = findOrCreateOrg();
    Bucket bucket = findOrCreateBucket(bucketNameGetter.getName(packetGroup), organization);
    final long time = packetCollection.getDateMillis();
    List<Point> points = new ArrayList<>();
    for (Packet packet : packetGroup.getPackets()) {
        Point point = pointCreator.createBuilder(packet).time(time, WritePrecision.MS);
        Collection<String> tagKeys = PointUtil.getTagKeys(packet.getClass());
        ObjectNode json = OBJECT_MAPPER.valueToTree(packet);
        for (Map.Entry<String, ValueNode> entry : PointUtil.flattenJsonObject(json)) {
            String key = entry.getKey();
            ValueNode prim = entry.getValue();
            if (tagKeys.contains(key)) {
                point.addTag(key, prim.asText());
            }
            if (prim.isNumber()) {
                // always store as float datatype because you can never change the type from int to float easily
                final Number value;
                if (prim.isBigDecimal()) {
                    DecimalNode decimal = (DecimalNode) prim;
                    value = decimal.decimalValue();
                } else {
                    value = prim.asDouble();
                }
                point.addField(key, value);
            } else if (prim.isTextual() || prim.isBinary()) {
                point.addField(key, prim.asText());
            } else if (prim.isBoolean()) {
                point.addField(key, prim.asBoolean());
            } else
                throw new AssertionError("This primitive isn't a number, string/binary or boolean! It's: " + prim + " class: " + prim.getClass() + " text=" + prim.asText());
        }
        points.add(point);
    }
    try {
        client.getWriteApiBlocking().writePoints(bucket.getName(), bucket.getOrgID(), points);
    } catch (InfluxException exception) {
        throw new PacketHandleException("Could not write points", exception);
    }
}
Also used : InstancePacketGroup(me.retrodaredevil.solarthing.packets.collection.InstancePacketGroup) Packet(me.retrodaredevil.solarthing.packets.Packet) Organization(com.influxdb.client.domain.Organization) ObjectNode(com.fasterxml.jackson.databind.node.ObjectNode) ArrayList(java.util.ArrayList) Point(com.influxdb.client.write.Point) Bucket(com.influxdb.client.domain.Bucket) ValueNode(com.fasterxml.jackson.databind.node.ValueNode) InfluxException(com.influxdb.exceptions.InfluxException) DecimalNode(com.fasterxml.jackson.databind.node.DecimalNode) Map(java.util.Map) PacketHandleException(me.retrodaredevil.solarthing.packets.handling.PacketHandleException)

Aggregations

PacketHandleException (me.retrodaredevil.solarthing.packets.handling.PacketHandleException)4 ObjectNode (com.fasterxml.jackson.databind.node.ObjectNode)2 ValueNode (com.fasterxml.jackson.databind.node.ValueNode)2 IOException (java.io.IOException)2 Map (java.util.Map)2 Packet (me.retrodaredevil.solarthing.packets.Packet)2 InstancePacketGroup (me.retrodaredevil.solarthing.packets.collection.InstancePacketGroup)2 JsonProcessingException (com.fasterxml.jackson.core.JsonProcessingException)1 DecimalNode (com.fasterxml.jackson.databind.node.DecimalNode)1 Bucket (com.influxdb.client.domain.Bucket)1 Organization (com.influxdb.client.domain.Organization)1 Point (com.influxdb.client.write.Point)1 InfluxException (com.influxdb.exceptions.InfluxException)1 ArrayList (java.util.ArrayList)1 CouchDbException (me.retrodaredevil.couchdbjava.exception.CouchDbException)1 CouchDbNotFoundException (me.retrodaredevil.couchdbjava.exception.CouchDbNotFoundException)1 CouchDbUpdateConflictException (me.retrodaredevil.couchdbjava.exception.CouchDbUpdateConflictException)1 JsonData (me.retrodaredevil.couchdbjava.json.JsonData)1 StringJsonData (me.retrodaredevil.couchdbjava.json.StringJsonData)1 DocumentResponse (me.retrodaredevil.couchdbjava.response.DocumentResponse)1