Search in sources :

Example 51 with AbstractTableConfig

use of com.linkedin.pinot.common.config.AbstractTableConfig in project pinot by linkedin.

the class LLRealtimeSegmentDataManagerTest method createFakeSegmentManager.

private FakeLLRealtimeSegmentDataManager createFakeSegmentManager() throws Exception {
    LLCRealtimeSegmentZKMetadata segmentZKMetadata = createZkMetadata();
    AbstractTableConfig tableConfig = createTableConfig();
    InstanceZKMetadata instanceZKMetadata = new InstanceZKMetadata();
    RealtimeTableDataManager tableDataManager = createTableDataManager();
    String resourceDir = _segmentDir;
    Schema schema = Schema.fromString(makeSchema());
    ServerMetrics serverMetrics = new ServerMetrics(new MetricsRegistry());
    FakeLLRealtimeSegmentDataManager segmentDataManager = new FakeLLRealtimeSegmentDataManager(segmentZKMetadata, tableConfig, instanceZKMetadata, tableDataManager, resourceDir, schema, serverMetrics);
    return segmentDataManager;
}
Also used : MetricsRegistry(com.yammer.metrics.core.MetricsRegistry) InstanceZKMetadata(com.linkedin.pinot.common.metadata.instance.InstanceZKMetadata) Schema(com.linkedin.pinot.common.data.Schema) ServerMetrics(com.linkedin.pinot.common.metrics.ServerMetrics) AbstractTableConfig(com.linkedin.pinot.common.config.AbstractTableConfig) LLCRealtimeSegmentZKMetadata(com.linkedin.pinot.common.metadata.segment.LLCRealtimeSegmentZKMetadata)

Example 52 with AbstractTableConfig

use of com.linkedin.pinot.common.config.AbstractTableConfig in project pinot by linkedin.

the class UpdateSegmentState method getAllTenantTables.

public List<String> getAllTenantTables() throws Exception {
    String tableConfigPath = "/CONFIGS/TABLE";
    List<ZNRecord> tableConfigs = _propertyStore.getChildren(tableConfigPath, null, 0);
    List<String> tables = new ArrayList<>(128);
    for (ZNRecord znRecord : tableConfigs) {
        AbstractTableConfig tableConfig = AbstractTableConfig.fromZnRecord(znRecord);
        if (tableConfig.getTenantConfig().getServer().equals(_tenantName)) {
            tables.add(tableConfig.getTableName());
        }
    }
    return tables;
}
Also used : ArrayList(java.util.ArrayList) AbstractTableConfig(com.linkedin.pinot.common.config.AbstractTableConfig) ZNRecord(org.apache.helix.ZNRecord)

Example 53 with AbstractTableConfig

use of com.linkedin.pinot.common.config.AbstractTableConfig in project pinot by linkedin.

the class PerfBenchmarkDriver method configureTable.

public void configureTable(String tableName, List<String> invertedIndexColumns) throws Exception {
    String jsonString = ControllerRequestBuilderUtil.buildCreateOfflineTableJSON(tableName, _serverTenantName, _brokerTenantName, _numReplicas, _segmentAssignmentStrategy).toString();
    AbstractTableConfig offlineTableConfig = AbstractTableConfig.init(jsonString);
    offlineTableConfig.getValidationConfig().setRetentionTimeUnit("DAYS");
    offlineTableConfig.getValidationConfig().setRetentionTimeValue("");
    IndexingConfig indexingConfig = offlineTableConfig.getIndexingConfig();
    indexingConfig.setLoadMode(_loadMode);
    indexingConfig.setSegmentFormatVersion(_segmentFormatVersion);
    if (invertedIndexColumns != null && !invertedIndexColumns.isEmpty()) {
        indexingConfig.setInvertedIndexColumns(invertedIndexColumns);
    }
    _helixResourceManager.addTable(offlineTableConfig);
}
Also used : IndexingConfig(com.linkedin.pinot.common.config.IndexingConfig) AbstractTableConfig(com.linkedin.pinot.common.config.AbstractTableConfig)

Aggregations

AbstractTableConfig (com.linkedin.pinot.common.config.AbstractTableConfig)53 ZNRecord (org.apache.helix.ZNRecord)10 Test (org.testng.annotations.Test)10 IdealState (org.apache.helix.model.IdealState)9 ArrayList (java.util.ArrayList)8 JSONObject (org.json.JSONObject)8 HttpVerb (com.linkedin.pinot.common.restlet.swagger.HttpVerb)7 Paths (com.linkedin.pinot.common.restlet.swagger.Paths)7 Summary (com.linkedin.pinot.common.restlet.swagger.Summary)7 Tags (com.linkedin.pinot.common.restlet.swagger.Tags)7 IOException (java.io.IOException)7 HashMap (java.util.HashMap)7 StringRepresentation (org.restlet.representation.StringRepresentation)7 BeforeTest (org.testng.annotations.BeforeTest)7 TableNameBuilder (com.linkedin.pinot.common.config.TableNameBuilder)6 Schema (com.linkedin.pinot.common.data.Schema)6 IndexingConfig (com.linkedin.pinot.common.config.IndexingConfig)5 KafkaStreamMetadata (com.linkedin.pinot.common.metadata.stream.KafkaStreamMetadata)4 JSONException (org.json.JSONException)4 SegmentsValidationAndRetentionConfig (com.linkedin.pinot.common.config.SegmentsValidationAndRetentionConfig)3