The new AMALTHEA Trace DataBase (ATDB) primarily used to represent a runtime trace in a database format. As part of the rework, ATDB can also be used to calculate derived metrics from the trace data it holds. Since it is intended to be used in conjunction with AMALTHEA, the calculable metrics are a subset of the metrics found in AMALTHEA. However, the database schema allows for arbitrary events, entities, properties, and metrics.
The database is stored as SQLite Database with extension .atdb. It provides the data tables and several views to make the data contained in the data base more user-accessible. For performance reasons, there are some indices to speed up the access times. The general database structure and views will be described in the following sections.
The database schema is defined completely static. Depending on the detail of data to be stored some of the tables are optional (they can be temporary to calculate metrics and do not need to be persisted). The following tables are defined:
The database schema as Entity Relationship diagram:
The core tables refer to all white tables from the picture, without them, the database would not be meaningful. So these are mandatory to contain datasets.
This table contains simple key-value pairs containing information about the trace, how it was recorded, how long it is, etc. It may also contain other information, e.g., the file name(s) of the related AMALTHEA model.
So far, commonly used names include the following:
The entity table contains information about all artifacts used in the database. This may include things that are not visible in the underlying trace, e.g., event chains.
In order to classify entities, this table contains datasets about their types. These types can be referenced in the entityTypeId column of the entity table.
Currently, the following type names are commonly used (also see BTF specification):
This table may contain more entity types as long as they are unique within this table.
Entities may be instantiated multiple times. In a trace this may be the time span between the start and terminate event of a runnable entity. For a periodic time stimulus an instance represents the nth occurence of the time stimulus. In order to differentiate between these instances this table holds two values which act as a unique identification for entity instances. They will be referenced from various entity-instance-specific tables.
Similar to entity types this table contains types of events that can be referenced from the event and trace event tables. This also represents the connection between which events shall be observed (the ones in the event table) and which events are actually in the trace (the ones in the trace event table). The commonly used event type names are the intersection between the events from AMALTHEA and the events from the BTF specification. Just like for entity types, this table may contain additional event types (unique).
Additional information about entities which may help to understand and analyze the trace can be provided in auxillary tables. They are represented in yellow in the above data base schema diagram. Some information in these tables can be derived from the trace, e.g., the runnable to task mapping, other data (like event chains and events) will be sourced from the original AMALTHEA model.
A table for all possible properties that an entity may have in the database.
Currently, the following property names with their types are used:
Here, the actual properties are set for entities. Multiple property values are represented with a strictly sequential counting number (sqcnr). In case of ordered multiple values, this acts as the index of the value within the list of values.
Observable event specifications directly sourced from an AMALTHEA model can be stored in this table. Since event sets are handled via multi-valued properties, this table directly corresponds the abstract EntityEvent meta-class from AMALTHEA. The description field is left out. The name, eventType, and entity columns directly match the AMALTHEA pendants. The subtype-specific references in the EntityEvents are abstractly represented by the sourceEntity in this table. The events contained in this table are referenced in event chain entities.
Derived information about entities and their instances which can be calculated (via metrics) based on an event trace may be stored in metric tables. In the above diagram these tables are colored in green.
Similar to Property, this table contains the name and dimension for metrics which are referenced by the other metric tables. Commonly used metrics per entity type are defined in the Requirements section of the AMALTHEA data model.
Contains entity-specific values for given metrics.
Contains entity-instance-specific values for given metrics.
Contains entity-specific values for given metrics. The sqcnr can be used to represent instances which are not differentiated by entity instances (e.g. by a constant sized time frame). Metrics like the average core load based on time slices can be stored here.
Depending on how much information shall be exchanged/contained in this data base the following tables do not have to be stored (they can be declared as TEMPORARY). They can, however, be quite usefull in calculating metrics about entities in a structured and comprehensive way. Once the metrics are calculated, there is no reference back to these optional tables. So, they do not have to remain in the data base. Since these tables represent a full event trace, omitting them can greatly reduce the size of the data base.
Opposite to the observable events, this table is used to store all events directly from a BTF trace (in timestamp ascending order). For multiple events happening at the same timestamp there is a strictly sequential counting number. All other columns of this table correspond to a typical event line in a BTF trace.
Derived information about runnable events for easier metric calculation. All values are derived from TraceEvent. The event counts per runnable instance stored in this table help to determine if the runnable instance is completely captured within the trace.
Derived information about process (task or ISR) events for easier metric calculation. All values are derived from TraceEvent. The event counts per process instance stored in this table help to determine if the process instance is completely captured within the trace.
Derived information about event chain instances for easier metric calculation. All values are derived from event chain entities (and their properties) and TraceEvent. This table connects the event chain instances in EntityInstance to their corresponding stimulus and response events (via Compound Foreign Key) in TraceEvent. Since event chains can be subject to EventChainLatencyConstraints and there are only two types of instances considered (age and reaction) this table holds information about whether the event chain instance is age, reaction, or both.
In order to better comprehend the contents of the database, SQLite offers the definition of views, which can be used to display the tables in a more readable way. With the help of views, the database contents can be presented in a standard database browser without knowing the meaning or significance of the table values – this is conveyed in views. This section will describe the views, which in turn can be understood as database tables, and how they are derived from the persisted tables from above. The how will also be given as SQL statements.
The core tables include MetaInformation, Entity, EntityType, EntityInstance, and EventType. Since these tables only contain a small number of columns which are comprehensible on their own, there are no views defined for the core tables.
The auxillary data tables contain more columns and refer to other tables to some extent. The following views are defined for ausillary tables:
This view shows the entity names and property names, with their corresponding values (multiple values are shown in one value entry, where the values are concatenated). In addition they also show the entity and property type names.
The corresponding SQL query to generate the view:
SELECT
(SELECT name FROM entity WHERE id = propertyValue.entityId) AS entityName,
(SELECT name FROM entityType WHERE id = (SELECT entityTypeId FROM entity WHERE id = propertyValue.entityId)) AS entityType,
(SELECT name FROM property WHERE id = propertyValue.propertyId) AS propertyName,
(SELECT type FROM property WHERE id = propertyValue.propertyId) AS propertyType,
(GROUP_CONCAT(CASE
WHEN propertyValue.propertyId IN (SELECT id FROM property WHERE type = 'entityIdRef') THEN
(SELECT name FROM entity WHERE id = propertyValue.value)
WHEN propertyValue.propertyId IN (SELECT id FROM property WHERE type = 'eventIdRef') THEN
(SELECT name FROM event WHERE id = propertyValue.value)
ELSE
propertyValue.value
END, ', ')) AS value
FROM propertyValue GROUP BY entityId, propertyId ORDER BY entityId, propertyId
The observable event specification refers to EventType and Entity. This view resolves the referenced ids and presents the events in an understandable way.
The corresponding SQL query to generate the Event view:
SELECT
name,
(SELECT name FROM eventType WHERE id = eventTypeId) AS eventType,
(SELECT name FROM entity WHERE id = entityId) AS entityName,
(SELECT name FROM entityType WHERE id =
(SELECT entityTypeId FROM entity WHERE id = event.entityId)
) AS entityType,
(SELECT name FROM entity WHERE id = sourceEntityId) AS sourceEntityName,
(SELECT name FROM entityType WHERE id =
(SELECT entityTypeId FROM entity WHERE id = event.sourceEntityId)
) AS sourceEntityType
FROM event
As a special case, event chains are also stored as entities, however, they never act as a subject or source entity in the trace event table. In order to better present and highlight them among all other entities, this view provides a comprehensive representation of all event chain entities.
The corresponding SQL query to generate the event chain entity view:
SELECT
name AS eventChainName,
(SELECT GROUP_CONCAT(name, ', ') FROM event WHERE id IN (SELECT value FROM propertyValue WHERE entityId = ecEntity.id AND
propertyId = (SELECT id FROM property WHERE name = 'ecStimulus'))) AS stimulus,
(SELECT GROUP_CONCAT(name, ', ') FROM event WHERE id IN (SELECT value FROM propertyValue WHERE entityId = ecEntity.id AND
propertyId = (SELECT id FROM property WHERE name = 'ecResponse'))) AS response,
(SELECT GROUP_CONCAT(name, ', ') FROM entity WHERE id IN (SELECT value FROM propertyValue WHERE entityId = ecEntity.id AND
propertyId = (SELECT id FROM property WHERE name = 'ecItems' ))) AS items,
(SELECT value FROM propertyValue WHERE entityId = ecEntity.id AND propertyId =
(SELECT id FROM property WHERE name = 'ecMinItemsCompleted')) AS minItemsCompleted,
EXISTS(SELECT value FROM propertyValue WHERE entityId = ecEntity.id AND propertyId =
(SELECT id FROM property WHERE name = 'ecMinItemsCompleted')) AS isParallel
FROM entity AS ecEntity WHERE entityTypeId = (SELECT id FROM entityType WHERE entityType.name = 'EC')
The metric views resolve the entity and metric ids into the corresponding names and displays them. The three metric views are thus very similar.
This view presents non-instance-specific entity metrics.
The corresponding SQL query to generate the entity metric value view:
SELECT
(SELECT name FROM entity WHERE id = entityMetricValue.entityId) AS entityName,
(SELECT name FROM entityType WHERE id =
(SELECT entityTypeId FROM entity WHERE id = entityMetricValue.entityId)
) AS entityType,
(SELECT name FROM metric WHERE id = entityMetricValue.metricId) AS metricName,
entityMetricValue.value
FROM entityMetricValue
ORDER BY entityId, metricId
A representation of the entity instance-specific metrics is given with this view.
The corresponding SQL query to generate the entity instance metric value view:
SELECT
(SELECT name FROM entity WHERE id = entityInstanceMetricValue.entityId) AS entityName,
(SELECT name FROM entityType WHERE id =
(SELECT entityTypeId FROM entity WHERE id = entityInstanceMetricValue.entityId)
) AS entityType,
entityInstanceMetricValue.entityInstance,
(SELECT name FROM metric WHERE id = entityInstanceMetricValue.metricId) AS metricName,
entityInstanceMetricValue.value
FROM entityInstanceMetricValue
ORDER BY entityId, entityInstance, metricId
This view shows entity metrics that are not grouped by entity instances but by metric instances.
The corresponding SQL query to generate the entity metric instance value view:
SELECT
(SELECT name FROM entity WHERE id = entityMetricInstanceValue.entityId) AS entityName,
(SELECT name FROM entityType WHERE id =
(SELECT entityTypeId FROM entity WHERE id = entityMetricInstanceValue.entityId)
) AS entityType,
(SELECT name FROM metric WHERE id = entityMetricInstanceValue.metricId) AS metricName,
entityMetricInstanceValue.sqcnr,
entityMetricInstanceValue.value
FROM entityMetricInstanceValue
ORDER BY entityId, metricId, sqcnr
These views are derived from optional tables. So they may not be persisted in the database file.
The trace event table itself is not readable like the BTF since all the references are ids. This view therefore offers a more BTF-like trace event table.
The corresponding SQL query to generate the TraceEvent view:
SELECT
traceEvent.timestamp,
traceEvent.sqcnr,
(SELECT name FROM entity WHERE id = traceEvent.entityId) AS entityName,
(SELECT name FROM entityType WHERE id =
(SELECT entityTypeId FROM entity WHERE id = traceEvent.entityId)
) AS entityType,
traceEvent.entityInstance,
(SELECT name FROM entity WHERE id = traceEvent.sourceEntityId) AS sourceEntityName,
(SELECT name FROM entityType WHERE id =
(SELECT entityTypeId FROM entity WHERE id = traceEvent.sourceEntityId)
) AS sourceEntityType,
traceEvent.sourceEntityInstance,
(SELECT name FROM eventType WHERE id = traceEvent.eventTypeId) AS eventType,
traceEvent.value
FROM traceEvent
This provides a filtered view on TraceEvent where the subject entity is a runnable.
The corresponding SQL query to generate the runnable instance TraceEvent view:
SELECT
timestamp,
sqcnr,
(SELECT name FROM entity WHERE id = entityId) AS runnableName,
entityInstance,
(SELECT name FROM entity WHERE id = sourceEntityId) AS sourceEntityName,
sourceEntityInstance,
(SELECT name FROM eventType WHERE id = eventTypeId) AS eventType
FROM (
SELECT timestamp, sqcnr, entityId, entityInstance, sourceEntityId, sourceEntityInstance, eventTypeId
FROM traceEvent WHERE
eventTypeId IN (SELECT id FROM eventType WHERE name IN ('start', 'resume', 'terminate', 'suspend')) AND
entityId IN (SELECT id FROM entity WHERE entityTypeId IN
(SELECT id FROM entityType WHERE name IN ('R'))
)
GROUP BY entityId, entityInstance, timestamp, sqcnr
)
This provides a filtered view on TraceEvent where the subject entity is a process (Task or ISR).
The corresponding SQL query to generate the process instance TraceEvent view:
SELECT
timestamp,
sqcnr,
(SELECT name FROM entity WHERE id = entityId) AS processName,
entityInstance,
(SELECT name FROM entity WHERE id = sourceEntityId) AS sourceEntityName,
sourceEntityInstance,
(SELECT name FROM eventType WHERE id = eventTypeId) AS eventType
FROM (
SELECT timestamp, sqcnr, entityId, entityInstance, sourceEntityId, sourceEntityInstance, eventTypeId
FROM traceEvent WHERE
eventTypeId IN (SELECT id FROM eventType WHERE name IN ('activate', 'start', 'resume', 'run', 'terminate',
'preempt', 'poll', 'wait', 'poll_parking', 'park', 'release_parking', 'release', 'boundedmigration',
'fullmigration', 'mtalimitexceeded')) AND
entityId IN (SELECT id FROM entity WHERE entityTypeId IN
(SELECT id FROM entityType WHERE name IN ('T', 'I'))
)
GROUP BY entityId, entityInstance, timestamp, sqcnr
)
These two views directly represent the instance trace info tables, except that the entityId is resolved to the name of the entity.
This view maps event chain, stimulus event, and response event entities to their corresponding names, which makes the EventChainInstanceInfo table more comprehensible.
The corresponding SQL query to generate the EventChainInstanceInfo view:
SELECT
(SELECT name FROM entity WHERE id = entityId) AS eventChainName,
entityInstance AS ecInstance,
stimulusTimestamp,
(SELECT name FROM entity WHERE id = (SELECT entityId FROM traceEvent WHERE timestamp = stimulusTimestamp AND
sqcnr = stimulusSqcnr)) AS stimulusEntityName,
(SELECT entityInstance FROM traceEvent WHERE timestamp = stimulusTimestamp AND sqcnr = stimulusSqcnr) AS stimulusEntityInstance,
(SELECT name FROM eventType WHERE id = (SELECT eventTypeId FROM traceEvent WHERE timestamp = stimulusTimestamp AND
sqcnr = stimulusSqcnr)) AS stimulusEvent,
responseTimestamp,
(SELECT name FROM entity WHERE id = (SELECT entityId FROM traceEvent WHERE timestamp = responseTimestamp AND
sqcnr = responseSqcnr)) AS responseEntityName,
(SELECT entityInstance FROM traceEvent WHERE timestamp = responseTimestamp AND
sqcnr = responseSqcnr) AS responseEntityInstance,
(SELECT name FROM eventType WHERE id = (SELECT eventTypeId FROM traceEvent WHERE timestamp = responseTimestamp AND
sqcnr = responseSqcnr)) AS responseEvent,
(CASE WHEN isAge AND isReaction THEN 'age/reaction' WHEN isAge THEN 'age' WHEN isReaction THEN 'reaction' END) AS latencyType
FROM eventChainInstanceInfo