Ytica gathers individual events from Twilio and transforms them into an easy to understand business oriented Analytics Data Model. This is a multi-stage process. The below scheme shows high level overview of the entire pipeline. Note that components not directly related to Ytica for Twilio are not shown in the scheme.
Twilio Listener API
Accepts incoming events from Twilio TaskRouter and distributes individual events to a swarm of Twilio Event Listeners.
Twilio Event Listener
Processes an individual events and stores them in Event Journal.
Stores all incoming events for further processing. Critical items such as timestamp and task ID is indexed. The original event is stored in an original form.
Event Journal is a temporary storage. Data from Event Journal are not available via an API nor via a user interface.
Periodically extracts events from Event Journal and for all Twilio tasks that were completed or cancelled creates a corresponding items in Analytics Storage. Also creates or updates all related data to the task.
Twilio Transformer cleans the data and unifies them with data received from other data sources.
Stores all data available for Analytics in Analytics Data Model structure. Analytics Storage is dedicated just to Ytica for Twilio. It may contain data from multiple data sources linked together to provide the best possible context to the conversations.
Analytics Storage contains near-realtime data.
Periodically extracts data from Analytics Storage and transforms them into data structure that is highly optimized for flexible analytics.
Deep Analytics Storage
Stores all data available for Analytics in Analytics Data Model structure. This is from the outside perspective the same structure as in Analytics Storage. Internally the data are stored differently to enable for operations that Analytics Storage cannot provide.
Deep Analytics Storage contains historical data that are updated daily or hourly depending on plan you choose.
Most of the infrastructure runs on different Amazon AWS infrastructure and services except for Deep Analytics Storage that uses a dedicated hardware for making Analytics affordable to as many customers as possible.