KPIs are now known as Behavior Indicators to more accurately describe their purpose.
Our Behavior Indicators were originally based on comparison to all Aware customers’ communication data. Going forward, the models are now trained solely on your organization's data, making the Indicator a benchmark against how your employees typically communicate, known as ‘Standard’. The goal of the Indicators is to highlight anomalies in particular sets of messages (org level, groups, topics, etc.), by comparing to your Standard baseline, which was created by determining the normal distribution of millions of messages. This model establishes a new baseline based upon your unique communication behavior, providing you more actionable insights by shining a light on anomalies in your organization’s conversations
What changed with the models?
To make the new changes more consumable, the [0-100] Sentiment and Health scale is now based on percentiles, allowing us to better understand the distribution of the message set. This means that your organization's normal conversation, or "Standard," is within the 25th-75th (median) percentile range. The Indicator tells you how a dataset of messages deviates from what is 'Standard' in your organization. For example, a Sentiment Indicator of 90 tells us that this dataset falls in the 90th percentile of your organization's communication, which is Above Standard and means 90% of typical conversation is lower in Sentiment than this data set. Indicators falling in the Above Standard and Below Standard range are considered outliers from the normal conversation in your organization.
This illustration shows how Standard is determined for your organization, by understanding the normal distribution of messages.
The Indicator is determined by taking a dataset of messages, like the Daily Organization roll up, and comparing it to the Standard distribution. The Indicator is created from the deviation of the distribution.
Why do my indicators drop or change significantly?
The Sentiment and Health Indicators are comparing your communication data to your own baseline, making Standard (or what is normal) the 25th-75th percentile. As an example, if you normally saw Indicators in the 90s historically, your normal or "Standard" is now calibrated to be the median, or [25-75]. So, while you may have been Above Standard compared to other organizations, that is now the baseline for your company’s communication. If your Indicator is in the Above Standard threshold, that dataset of messages is higher Sentiment or Health than how your organization typically communicates. If your Indicator is Below Standard, that dataset of messages is lower Sentiment or Health than how your organization usually communicates.
Why did we do this?
Every organization has a unique communication behavior. Comparing to your own baseline gives you an accurate pulse check on how your organization is doing compared to how they typically communicate. By making the models reflective of your organization, we can provide more precise Indicators that shed light on areas of your organization that aren't acting like themselves. This change will allow us to build in more applicable benchmarks, e.g.: industry, department, frontline, organization size, etc. in the future. Our plans include building benchmarking capabilities against different verticals.