Signal
Datablit Signals are aggregation(eg. count, sum, avg etc) metrics. Signals are computed over a grouping id called entity id eg. flight_id. device_id, user_id
Eg. No of user orders in last 24hrs
Following basic terminologies will help you to create and understand signals.
Signal key
It's unique lowercase string that you can define to use as signal key. We recommed to use shorter length to save memory. You can refer this in rule. eg. orders_24h
Entity ID
It's grouping key on which signal aggregtion happens. eg. user_id, flight_id
Aggregation window
An aggregation window defines the time period over which signals are grouped. For example: if you want to count the number of orders in the last 24 hours, the aggregation window is 24 hours.
Freshness
It represents the maximum time difference between an event’s occurrence and signal computation — a measure of how up-to-date your signal is relative to when it was generated. Example: A user opens an app at 8:30, 8:40, 9:00, 9:10, 9:30, etc. If you want to compute the total number of sessions in the last hour and evaluate this every hour (at 8:00, 9:00, etc.), the signal’s freshness would be 1 hour. This means the computed session count will be 1 hour.
In short, the aggregation window is about grouping and freshness is about delay.
Signal Types
Based on aggregation window and freshness requirements, you can choose how you want to create a signal.
**Real time for high freshness(< 1 minute) and upto 24 hours aggregation window ** - These are ideal for scenarios requiring immediate data updates, making it a solid option for business to personalise at scale and take immediate action. Example: Number of sessions in the last hour
Batch processing works best when you need in depth analysis at set intervals - As the name suggests, used to process data in batches. If you need to aggregate a large number of data points over longer periods and can tolerate a few hours of freshness. Example: Number of orders in the last 1 year with 12-hour freshness
Comparison of Signal Types
| Signal Type | Aggregation Window | Freshness | Computation | Example Use Case |
|---|---|---|---|---|
| Realtime Signal | Short (e.g., 1 hour) | High (< 1 minute) | For each event | Number of sessions in the last hour |
| Batch Signal | Long (e.g., 1 years) | Lower (few hours) | After specific duration (e.g., every 6 hours) | Number of orders in the last 1 years (12-hour freshness) |
Realtime Aggregation Types
It refers to how data points or events are grouped. Realtime signals support two types of aggregation:
- Count: The number of times an event occurs
- Sum: Sum a specific field name from events
| Aggregation Type | Signal |
|---|---|
| SUM | total_order_amount |
| STATIC | plan_type |
| COUNT | no._of_order |
Signal Examples
| Signal | Entity id | Signal type | Aggregation window | Freshness |
|---|---|---|---|---|
| Number of user orders in last 24hrs | user_id | Realtime | 24 hours | High(< 1 minute) |
| Number of flight booked in last 4hrs | flight_id | Realtime | 4 hours | High(< 1 minute) |
| Total user order amount in last 30 days | user_id | Batch | 30 days | Low(>= 1 hour) |