We propose an unsupervised approach to discover frequent activities form email logs. Our approach allows the detection of multiple activities per email and the automatic generation of their names while considerably reducing human intervention.
We evaluated the proposed approach on real emails belonging to Enron public dataset (https://www.cs.cmu.edu/~enron/). To this end, we were based on the emails of 5 employees. To visualize the obtained results, we have implemented a visualization tool that generates for each employee a figure resuming his main activities.
In this document, we provide the visualization results that we have obtained for the employees considered in our experiments. We present also details on the results that we obtained in the evaluation phase. In such phase, we carried out two evaluation types : (1) Evaluation based on annotated emails where we used emails of two employees, and (2) Extended evaluation where we used emails of 5 employees.
The following table describes the data that we use in the evaluation phase. For each employee, it shows : (1) his/her name, (2) his/her role (3) his/her business domain in Enron, and (4) the number of his/her sent emails.
| ID | Employee Name | Role | Domain | Number of analyzed emails |
|---|---|---|---|---|
| E1 | John Forney | Manager | Real time Trading Desk | 331 |
| E2 | Shirley Crenshaw | Employee | Administrative Assistant | 342 |
| E3 | Monika Causholli | Employee | Analyst Risk Management | 375 |
| E4 | Daren Farmer | Manager | Logistics Manager | 589 |
| E5 | Vince Kaminski | Manager | Risk Management Head | 1940 |
In the overall of this section, a frequent activity threshold means the mimimum occurrence number of an activity that defines if it is frequent or not.
We selected here all the emails sent by the two employees E1 and E2 and we manually annotated the list of discussed activities. We ran then our algorithm to discover frequent activities of both employees. We used here two measures to evaluate each discovered activity :
We calculate then the average of each measure. We varied the frequent activity threshold in the interval [5,12] to study the evolution of each average. Figure below shows this evolution. We can notice that the recall average and the accuracy average vary respectively in the following intervals [0.86, 0.88] and [0.92, 0.96] . This shows that our algorithm is able to detect relevant activities from emails with good recall (>= 0,86) and accuracy (>= 0,92). Additionally, the performances of such detection would be better by increasing the threshold of frequent activities (especially for accuracy performance). This validates one assumption that we suppose during our work : One employee executing frequent activities likely use close wordings to express them. By discovering the common substructures (patterns) characterizing such close wordings, activities can be recognized from emails.
We aim here to evaluate the relevance of the discovered activities, wich means the fraction of relevant activities among all the retreived activities. We extended the evaluation data set to the emails of 5 employees as the relevance metric does not require annotated emails. We ran our algorithm on the emails of each employee. Then, we annotated the obtained activities (relevant or not). We evaluated, for each employee and then for all employees, the evolution of the relevance metric value when varying the frequent activity threshold in the interval [5,12]. Figure below shows the obtained results.
Red curve in the figure shows the evolution of the relevance metric value considering the activities of all employees. Such value varies in the interval of [0.75,0.90]. It increases when increasing the frequent activity threshold. This result would validate another assumption in our work ; One employee executing frequent activities likely use close wordings to express them. This fact would bring a kind of regularity at the level of such wordings. This regularity would result in the appearance of frequent substructures (patterns) reflecting relevant activities. As consequence, patterns frequently shared would reflect relevant sub-structures characterizing activities, while what is shared at low frequency is likely to present noise (irrelevant information) (This is why the relevance ratio decreases when decreasing the threshold of frequent activities).
The other curves (E1, E2 , E3 , E4 , E5 ) show the evolution of the relevance metric value for each employee. We can notice that the slope of these curves is almost positive while its value varies from one employee to another . For instance, it is close to zero with E1, E2 and considerably increases with E5 . This can be explained by the variety of the job nature of these employees and the purpose to it they send their emails; E1 and E2 have regular activities and most often use their emails to introduce them. On the contray, E5 can send emails to discuss general topics in relation to Enron strategies (without introducing specific activities), which can inrease the appearance of random combination of words .
