Data Corpus

We included in this webpage the original data set used and the results obtained by applying track before detect. bellow is the description of the dataset. N.B:Data are ordered, timestamped, single-valued metrics. All data files contain anomalies, unless otherwise noted.

Real data

• realAWSCloudwatch/

  AWS server metrics as collected by the AmazonCloudwatch service. Example metrics include CPU Utilization, Network Bytes In, and Disk Read Bytes.

• realAdExchange/

  Online advertisement clicking rates, where the metrics are cost-per-click (CPC) and cost per thousand impressions (CPM). One of the files is normal, without anomalies.

• realKnownCause/

  This is data for which we know the anomaly causes; no hand labeling.

  • ambient_temperature_system_failure.csv: The ambient temperature in an office setting.
  • cpu_utilization_asg_misconfiguration.csv: From Amazon Web Services (AWS) monitoring CPU usage – i.e. average CPU usage across a given cluster. When usage is high, AWS spins up a new machine, and uses fewer machines when usage is low.
  • ec2_request_latency_system_failure.csv: CPU usage data from a server in Amazon's East Coast datacenter. The dataset ends with complete system failure resulting from a documented failure of AWS API servers. There's an interesting story behind this data in the Numenta blog.
  • machine_temperature_system_failure.csv: Temperature sensor data of an internal component of a large, industrial mahcine. The first anomaly is a planned shutdown of the machine. The second anomaly is difficult to detect and directly led to the third anomaly, a catastrophic failure of the machine.
  • nyc_taxi.csv: Number of NYC taxi passengers, where the five anomalies occur during the NYC marathon, Thanksgiving, Christmas, New Years day, and a snow storm. The raw data is from the NYC Taxi and Limousine Commission. The data file included here consists of aggregating the total number of taxi passengers into 30 minute buckets.
  • rogue_agent_key_hold.csv: Timing the key holds for several users of a computer, where the anomalies represent a change in the user.
  • rogue_agent_key_updown.csv: Timing the key strokes for several users of a computer, where the anomalies represent a change in the user.

• realTraffic/

  Real time traffic data from the Twin Cities Metro area in Minnesota, collected by the Minnesota Department of Transportation. Included metrics include occupancy, speed, and travel time from specific sensors.

• realTweets/

  A collection of Twitter mentions of large publicly-traded companies such as Google and IBM. The metric value represents the number of mentions for a given ticker symbol every 5 minutes.

Artificial data

• artificialWithAnomaly/

  Artificially-generated data with varying types of anomalies.

Results

To facilitate the interpretation of the results after using track before detect, we used three types of figures (scatter, density_heatmap and line);A CSV file is generated for each analyzed using a case where you can find the calculated anomaly score

• artificialWithAnomaly
• realAdExchange
• realAWSCloudwatch
• realKnownCause
• realTraffic
• realTweets

below is a mix of the results from existing methods like Gaussian Mode, Regression Model, KNN, LOF, STREAM, INFLO, Autoencoders

• folder