Investigating with Activities

Communication is any activity where two entities, such as people or processes, exchange information. When analyzing a communication, the order and context are important as well as the content.

Understanding Activities

Activities analysis is very useful in HTTP activities, where multiple requests occur. For example, a website with various frames all from the same source (for example, msn.com and its news stories and articles that are stored on MSN-controlled servers) will all need to be requested via separate HTTP GET requests. It is quite possible that multiple GET requests will occur simultaneously (a feature generally known as "HTTP pipelining"). The goal of the Activities analysis algorithms is to sort each of these requests into its own activity, with a way to see whether they all came from the same session (as shown by the session ID). This gives you the ability to see causal relationships in complex protocols such as HTTP.

The Awake UI displays activities as dialogs that show the request and response messages in the context of applications involved. To increase the accuracy of the analysis and to clarify the implications of an activity, Awake presents these application-layer exchanges in the order in which they occur, in the context of the session they are part of. The resulting sessions and activities are stored, and available to view. You can download these as standard packet capture (PCAP) files. See: Understanding the Device Entity and the Entity Tracker.

Activity Templates and Details

The Awake Console displays activity details in a format tailored to each protocol that displays key information for the highest level protocol parsed within the activity. The format is given by a template that is designed to show at a glance the activity that occurred. For example, a partial details screen for an HTTP activity has this format.
Figure 1. Activities Detail

The other fields (Session ID, Start Time, and so forth) have the same format, regardless of the application layer. Application-tailored templates are available for some of the application-layer protocols:
  • HTTP
  • DNS-UDP
  • ICMP

The remaining protocols appear with more generic network/transport layer IP, TCP, or UDP templates. Tailored templates for the other application-layer protocols are planned.

Activity templates present data to show you the content and effect of the communication. The templates do not attempt to specify whether that behavior is legitimate or not. In this release, you can use a combination of device Risk Analysis information, activity-level communication, and knowledge of your environment to help determine if a particular communication is legitimate.

You can drill down from the activity list into activity details. The details display the flow of communication between two systems. You can view a message's payload data in ASCII, hex, or UTF-8 format. Finally, you can download a Packet CAPture (PCAP) file and then use your favorite packet analysis tool for in-depth analysis.

For a listing of the templates for the parameters, see: Activity Protocol Templates.

Devices and Activities

Unlike devices, activities are not entities, and Awake does not attempt to associate activities with risk analysis, or with where they originate. In general, to link devices to activities, follow steps like these:
  1. Find devices with Risk Scores above a certain threshold (e.g. device.risk_score > 30).
  2. Review the device details for an interesting subset of these devices.
  3. Locate risk-triggering artifacts within the device details.
  4. Refine your query to look for a specific risk-triggering artifact.
  5. Switch to the Activities view.
  6. Review the existing activities or refine the query further to reduce the size of the result set.
  7. View the Activities Summary to review the packet data.
  8. Download a Packet CAPture (PCAP) file associated with an activity of interest.
  9. Analyze the PCAP file with your tool of choice.

By following this process, you can narrow your review to a targeted PCAP file from a specific device.

Not all activities will have associated devices on your network. New devices on your network can have activities data before entity tracking has created a device entity or before risk analysis has run. Some application-layer protocols, such as broadcast or DHCP, do not lend themselves to associations with particular devices. In both of these example situations, you will see activities without associated devices.

Exploring Activity in the Workbench

The workbench provides three views of query results (pivoting to show query-related devices, domains, or activities), and it gives access to the Query Panel, which enables advanced users to construct queries from scratch to locate information about specific aspects of the network. It also allows the user to export query results as a CSV or TSV file.

This section describes the Workbench fields and controls. It includes:

The use of the Activities Tab is described in Investigating with Activities.

Narrowing Query Results Using Feature Summaries

You can narrow the results of a query by filtering the results based on Device, Domain, or Activity parameters. To do this, add parameters using the Feature Summaries choice from the AML Suggestions panel.

Note that filters take effect only once you have executed a query and have query results, so be sure to perform a preliminary query before applying filters. A good strategy is to make your initial query broad, then use filters to narrow those results.

The Feature Summaries choice breaks things down into Device Summaries, Domain Summaries, and Activity Summaries. Here is what you see with Device Summaries selected:
Figure 2. Filters Device

If you select one of the query-narrowing items, your total query results will be narrowed accordingly. In the following image, the query is filtered by Operating System Version with 7, XP, and 10 as selected values. Note that Awake has not only filtered the results, but has also written out the query expression at the bottom of the screen.To add it to the current query as a filter, click + Add to Query.
Figure 3. Filtered Results

 

Understanding How Feature Summaries Work

Each Feature filters the entities found within a time window. You use the time controls (timeline, time menu, or range picker) to specify the time window. For details on the time window, see Using Time to Focus Query Results.

You can reduce the result set by adding expressions to a query. You can add expressions to the query bar by:
  • choosing filters from the AML Suggestions / Feature Summaries panel as described above.
  • adding artifact-value pairs from the Summary panel.
  • entering expressions directly into the query bar.

Using the query expressions, you can build a filter that further refines the query and timeline results.After modifying a query, remember to refresh the results by clicking the query button.

Feature Summaries Reference

The Feature Summaries menu has the following components:
  • Value
  • Count
  • % of Entities
  • Query string preview
  • Back to All Summaries button
  • ** +Add to Query** button
Using the Feature Summaries menu, you can:
  • Edit the query string. Use the Query string preview to adjust your selection and the expression you plan to add to the query. You can remove a value by selecting the value and clicking (x); this also clears its corresponding checkbox. You can also select NOT to find the complement of a value (everything not matching that value).
  • Click Back to All Summaries to go back a step and discard your choices of features.
  • Click + Add to Query. The system adds your query as the last expression in the query panel. Note that you can also edit queries in the query panel directly.

Using Time to Focus Query Results

Although many elements of query results are time-independent (such as IP addresses, device names, operating systems), activities are always time-specific, so the display of query results in the Workbench is always filtered by time. This topic describes ways to adjust the timeframe of your query results.

For more detailed information about query construction in general, see Adversarial Models.

Time Controls

The Workbench includes three areas where you can specify time elements. These are the time-range picker, the timeline, and the query panel.
Figure 4. Time Controls

All three controls allow you to adjust the time range of your query:
  • Select a time range from the time-range picker menu; for example, Last 1 Hour.
  • Type values directly into the time range fields to create a custom range.
  • Use the arrows to expand the timeline window. This sets the time-range picker to Custom values.
  • Use the mouse to select part of the timeline to zoom in on a narrow piece of the range. Here is a typical timeline display:
    Figure 5. Timeline Before Selection

    Here is the timeline while a selection is being made. The dark segment is the selection:
    Figure 6. Timeline During Selection

    Here is the timeline after the selection is made.
    Figure 7. Timeline After Selection

When you first open the Workbench, it searches for results within the time shown on the time picker. You can choose among seeing the Devices, Domains, or the Activities within that time period by clicking the appropriate tab. To submit a new query, you can either:
  • press ENTER on your keyboard while focus is in the query panel, or
  • change the time parameters and click Search.

In the query panel, you can specify device.first_seen and device.last_seen query parameters (manually or using the Feature Summaries panel). Unlike the time-range picker and the timeline, these query parameters filter the results without adjusting the timeline. This means that if you specify a device.first_seen or device.last_seen time that is outside of the range in the time picker, the query will not produce a result because it is searching over a time range that is too narrow.

How the Timeline Controls Work with Current Time

If you use the time-range menu to set the time window, the time element of your queries is relative to the current time when the time range is set. If you then take time to build a query or examine query results, the current time used to set the time window when you clicked Search will have moved into the past.

To pick up the latest results, you can refresh your browser view. Or, if you update your query expression and click Search, the Workbench uses a new value for the current time and updates your results accordingly.

To fix your results to a particular time window, use the timeline to select specific blocks. You can also enter a time manually in the time-range picker. When you select from the timeline or enter a time manually, the time-range menu changes to Custom. This removes the current time value from your query, so a refresh would not retrieve new results.

Capturing the Current Time as a Static Value

There may also be times where you want to bookmark a particular moment so that it will not be cleared when the current time is refreshed. To capture the current time, use the now function. An example: device.last_seen <= now.

Enter a Time Manually

You can enter a time manually using your keyboard by typing it into the time-range picker using the format YYYY-MM-DD hh:mm:sss.If there is a problem with the format you enter, the system blocks the field in red.

You can also manually add a time constraint to the query in the query panel: {% include "include_time_format.md" %}.

Using the Query Language

The query panel is always available at the top of the screen. The dropdown menu at top right will show the type of query: Search for simple keyword searches, or AML for full queries.

Simple Search

To run a simple search:
  1. Click on the query type dropdown menu and select Search.
  2. Click in the query panel next to the Search icon.
  3. Start typing search terms into the query panel. Your search results appear just below the query panel as soon as Awake finds matches, and will be updated automatically as you edit your search string. You can narrow down the results with the Search Filters pane to the right of the search results. For details, see Searching and Query.

AML Queries

An AML query consists of at least one expression. An expression consists of a property, a comparison operator, and a value. You can combine any number of expressions in a query.

To understand the syntax of the Awake query language, see the Awake Adversarial Modeling Language (AML) Reference.

Query Execution "In-process" Indication

When Awake is processing your query, the round blue button will show an X icon, and if you hover over it, a tooltip will say "Query is executing. Click to cancel."
Figure 8. Query Execution In-Progress

Interactions of Timeline and Query

Each Feature filters the entities found within a time window. You use the time controls (timeline, time menu, or range picker) to specify the time window. For details on the time window, see Using Time to Focus Query Results.

As you add expressions to a query, you reduce the result set. You can add expressions to the query bar by:
  • choosing filters from the AML Suggestions / Feature Summaries panel,
  • adding artifact-value pairs from the Summary panel, or
  • entering expressions directly into the query bar.

Using the query expressions, you build a data sieve that further refines the query and timeline results.

Expression Format

An example of a simple expression looks like this:

device.ip == 10.0.0.4

The left-hand side of the expression is a property. The right-hand side is the value you query for. The comparison operator is placed between the left and right side. In this example, the property is device.ip, the comparison operator is == (double equal sign), and the value is 10.0.0.4. This expression searches for devices with an IP address property of 10.0.0.4.

 

You can use logical operators to combine expressions into more complex queries.

device.ip == 10.0.0.4 && device.browser == "Chrome"

 

In this example, the logical operator && (and) searches for devices with a 10.0.0.4 address and using the Chrome browser. You can combine an unlimited number of expressions with logical operators.

 

To improve readability, you can use parentheses to define your expressions' boundaries. Using the previous query as an example, you can use parentheses as follows:

(device.ip == 10.0.0.4) && (device.browser == "Chrome")

Here is another example:

(device.risk_score >= 5) && (device.ip == 10.200.104.5)

 

You can also add a negation operator (!) before an expression:

!(device.ip == 10.0.0.4 || device.browser == "Chrome" || device.threat.ip.count >= 5)

 

You can use arithmetic operators on the right-hand side of an expression. You can also use a single property on the left side. All of these queries are equivalent:

  • device.risk_score < 5
  • device.risk_score < (2 + 3)
  • 5 > device.risk_score

When you combine properties in this way, the left and right-hand side of the expression must evaluate to the same type. Both device.risk_score and device.threat.ip.count are Number types. Consider this failing example:

device.risk_score < (device.operating_system + 3)

In this example, you cannot add the string returned from device.operating_system and the integer 3, so the query fails.

Specifying Properties and Values

You can use the query panel to search for device entities or activities. To list the query parameters available in your current timeline, use the Filters panel. You can also enter query expressions manually. For a full list of all parameters available in the Awake product, see the Awake Adversarial Modeling Language (AML) Reference.

The value on the right-hand side of any expression must evaluate to the same data type as the left-hand side. Queries using the string type are case insensitive. So, a query for device.operating_system == "ios" and device.operating_system == "iOS" return the same result set.

You must enclose a value string within quotes; for example, "value" is correct. Values such as IP addresses, integers, unknown, and property values, do not need quotes, such as:
  • device.ip == 192.168.0.1
  • device.risk_score >= 30

An unknown value means that the value has not been defined. To search for unknown values, specify the unknown keyword without quotes, for example, this expression: device.browser == unknown. You can also filter for values specified as an empty string, for example, this expression: device.characteristic.tls.server.certificate.subject.organization == "".

Cross-table Queries Explained

You can combine cross-table queries with && (AND), but not a logical OR. This is valid:
activity.protocol == "smb" && device.ip != 10.10.10.10
This is not:
activity.protocol == "smb"
device.ip != 10.10.10.10

A more subtle case is the query activity.ip.source != device.ip.

This query initially parses as valid, but once the user attempts to execute the query the following error is displayed "Predicates on different entities can only be combined with &&. Try using parentheses to group your predicates separately." The query in substring activity.ip.source != device.ip in range [0, 31) does not execute, for example. Do not attempt to build queries that directly compare values across activity, device or domain query terms. An example of a query using using both activity and device terms that can be used is the following: activity.ip.source == 10.10.10.10 && device.name == "Leos-macbook".

Specifying Functions in Queries

Here are the functions that Awake supports:
  • count: Provides the length of the array being queried. For example: count device.browser.all == 0 finds devices on your network without a browser installed, generally considered a potential "red flag."
  • domainFuzz: Provides an algorithm-based permuted search input based on the domain name that you enter. This finds similar-looking domain names that attackers can use for phishing and other attacks. For example: domain.name in domainFuzz "awakesecurity.com".
  • typoSquat: Provides an algorithm-based search input of possible domains that are nearly the same as the domain name you enter. For example: domain.name in typoSquat "awakesecurity.com".
  • now: Provides the current timestamp in a form that can be folded into a broader query. For example: device.last_seen <= now, which establishes a bookmark when it is executed. Otherwise, "now" is recalculated with every iteration of the query. For details, see Time Controls.

Regular Expression Queries

The query panel supports only the PostgreSQL 8.3 POSIX Regular Expressions syntax for like queries; you cannot use SQL-like queries. You can only use regular expressions to search for singleton string literals. Searching lists of string literals is not supported. To build a query using a Regular Expression (REGEX), use this format:
parameter like r/regex-string/

The regex-string is any PSQL 8.3 POSIX regular expression. Using this format, you could search for device names that contain a literal . (dot) in the initial position by entering this:

device.name like r/^\./ 

The following table illustrates some of the other common metacharacters you can use with POSIX Basic syntax:

 
Metacharacter How to Use It
[ ] A bracket expression matches any single character contained in the brackets. So, [a-z] matches any of 26 characters in the lowercase alphabet.
. Period or dot matches any single character. If you use the dot inside a POSIX bracket expression, it is a literal match. For example, x.z matches xyz but [a-z] matches x, ., or z.
^ The caret matches the starting position in a string.
[^ ] Excludes the characters in the brackets. So [a-z] matches any character that is not in the range of the lowercase alphabet.
$ Matches the end position of a string.
( ) Defines a marked subexpression or block capturing group. You can have 1-9 sub-expressions and reuse each by identifying it with \n in your expression.
\n Matches what the _n_th marked subexpression matched.
* Matches the preceding element (a character, bracket expression, subexpression) zero or more times. So x*y matches both the xxy and the xxxy strings, while (la)* matches the lalalalala string.
{m,n} Limits the matches for the preceding element. Find at least m but not more than n. So, (la){2,3} matches lala but not the lalalala I can't hear you string.

This means that you can use PSQL 8.3 POSIX syntax to do searches such as:

Query: device.name like r/^L/

Looks for: Strings that begin with L.

Matches: Linas-MacBook-Pro-2., Lelands-MacBook-Pro-2.

Query:device.name like r/nas/

Looks for: The substring nas.

Matches: Linas-MacBook-Pro-2.

Query:device.name like r/1$/

Looks for: Strings that end with 1.

Matches: ebf68357-52b5-d4af-7e0a-7f3ea224bbb1, 3fe03688-9cfa-5f80-dfc1-cc2736e796c1

Query:device.operating_system.version like r/10.*1/

Looks for: Devices that contain the string "10" eventually followed by a "1".

Matches: 10.11, 10.11.6, 10.21.3

Query: device.characteristic.http.request.host.stemmed like r/^[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.?$/

Looks for: Find HTTP requests destined for IP addresses.

You can only use the like operator with string types.

Warning: Awake does only basic checking of your REGEX format; It does not ensure that your REGEX is valid. You must ensure your REGEX is correct. Invalid REGEX expressions do not return results and do not necessarily return an error.

Forming Risk Level Queries

You can copy Risk-triggering artifacts and values to form new queries or expand existing queries. You can also manually enter query parameters in the query panel. The advantage of using the Filters panel is that it restricts your searches to known matches in your results.

Copy a Fingerprint-level Expression to a Query

If you encounter a Fingerprint artifact with a value you want to search for, you can query using it. You can create a new query based on the risk level or you can copy it to an existing query. To copy a Fingerprint to a new query:

  1. Query for a device.
  2. Locate a Characteristic attribute that interests you.
  3. Hover the cursor over the attribute.
    The Workbench displays the secondary menu.
    Figure 9. Secondary Menu

  4. Open the menu and choose New Query.
A new query with the value appears in the query panel:
Figure 10. New Query Panel

You can also choose Add to Query to extend an existing query. In this case, you must choose how the added value relates to the original query.

 
Menu Item Definition
AND Adds an && to the query, for example,
&& device.characteristic.tls.version.chosen == "SSLv3.0"
AND NOT Adds an && and a !, for example,
&& !(device.characteristic.tls.version.chosen == "SSLv3.0")
OR Adds an || operator, for example,
|| device.characteristic.tls.version.chosen == "SSLv3.0"
OR NOT Adds an || and a !, for example,
|| !(device.characteristic.tls.version.chosen == "SSLv3.0")

Manually Use Query Parameters

When querying for Characteristic artifact values, you can query for a specific Risk score or for the computed Risk score for that attribute. For example, you can query for a specific Characteristic artifact value:

device.characteristic.dns.request.question.name.stemmed like r/google.com/

Or you can query for the computed score of this risk artifact.

device.characteristic.dns.request.question.name.stemmed.score > 5

The table below shows how you could construct a query for these attributes or their computed values:

 
Description Format
Attribute alone characteristic artifact, for example
device.characteristic.dns.request.question.name.stemmed
Occurrences of this key/value on this device. Appears as the "This" percentage value on the summary panel. fg_count, for example
device.characteristic.dns.request.question.type.fg_count
Occurrences of this key/value across all devices. Appears as the "All" value in the summary panel. bg_count, for example
device.characteristic.dns.request.question.type.bg_count
Number of devices that observed this key/value. coverage, for example
device.characteristic.dns.request.question.type.coverage
You can use coverage queries to find devices which have some coverage greater than zero, for example:
device.characteristic.dns.request.question.type.coverage > 0
A computed value representing for this individual attribute. risk_score, for example
device.characteristic.dns.request.question.type.score
How this attribute's score ranks among all entities. percentile, for example
device.characteristic.dns.request.question.type.percentile

Specify Date/Time Values

If you are working with time properties such as device.last_seen or device.first_seen, do not use the == (double equal sign). Instead, use with the < (less than) or > (greater than) operators. Awake supports the ISO8601 RFC3339 date-time format with nanosecond support. The specification details the following format:

YYYY-MM-DDThh:mm:ss.sTZD

For example, to search for devices seen since 7:20 PM on June 11th, 2018, UTC, you would use this timestamp:

device.last_seen >= 2018-06-11T19:20:00.00Z

The fields in the ISO8601 RFC3339 date-time format are:

 
Field Description
YYYY Four-digit year.
MM Two-digit month (01=January, etc.).
DD Two-digit day of month (01 through 31).
hh Two-digit hour (00 through 23) (AM/PM not allowed).
mm Two-digit minute (00 through 59).
ss An optional two-digit second (00 through 59).
s Optional one or more digits representing a decimal fraction of a second.
TZD Time zone designator (Z or +hh:mm or -hh:mm). Z indicates Universal Coordinated Time (UTC).

The Workbench returns the results displayed in equivalent UTC time.

List Membership

You can use the query language to search for values in lists. For example, you can use CIDR notation to locate IP addresses in a range:

device.ip in 100::/64

You can also explicitly search for values within a list. For example, this query searches for any match in a list of versions.

device.operating_system.version in [ "10.11","10.11.1","10.11.2","10.11.3","10.11.4" ]

List membership queries do not take the order of the elements into account. To find an empty list, you can use []:

device.operating_system.version in []

Queries that Combine Devices and Activities

If you combine a device.* parameter with an activity.* or a domain.* parameter, it has implications for some of the UI behavior. This topic discusses those implications.

Device sorting functionality disabled in some situations.

By default, the Devices tab sorts by Risk Score in descending order. If you are using only device.* paths in a query, you can sort on the Risk Score column of the Devices tab. If you add activity.* parameters to the query, this sorting is disabled.

Not all operators are supported.

Note:If you combine a device.* parameter with an activity.* or a domain.* parameter, you can only combine them with an && (logical AND) operator; the || (logical OR) is not supported with these two parameters in combination. You can, however, combine multiple parameters of the same type with a || (logical OR); for example device.name == “frank-mac” || device.name == “bob-Iphone”.

See also:

Troubleshooting Your Queries

This section contains some tips for troubleshooting common problems you may run into when submitting queries.

If Your Query is Improperly Formed

As you type in the query panel, the console checks the syntax. If there is a syntax error, the Search button appears in red and dotted red underlines appear under potential errors.
Figure 11. Bad Syntax

Hover over the error to get information about the problem.
Figure 12. Trouble Hint

If Your Query Takes a Long Time or Times Out

If the load on the platform is high, your query may take a long time, or even time out. In this case, try submitting the query again.

If a Query Returns No Results

If a query cannot find any results, your time window may be too small. You will see a message suggesting that you increase the window.
Figure 13. Increase Time Window

If you are confident your query is correct, and that there should be data returned, contact your Awake Systems Engineer to help you troubleshoot the issue.

Saving Your Skill Definitions

You can customize Awake to your needs. Once you find a query that does what you need, you can save it and retrieve it to use again and again.

To save a query:
  1. Click the Disk icon to the right of the AML dropdown menu. The Save Skill dialog box appears.
    Figure 14. Saved Skill Button

  2. Enter a Title. In this example, we use Ransomware.
  3. You can enter a Reference Identifier. This is optional. In this example, the Reference Identifier is recipes.ransomware.
  4. Click Save.
    Figure 15. Saved Skill

Analyzing Activities

Activities are instances of application-layer communications on your network. Each type of activity has a specific protocol, such as HTTP and HTTPS for Web browsing. You can identify activities by querying for specific protocols, and you can view the details of an activity, for example to see the messages that were exchanged. This overall view of activities is called ActivityIQ™, because it gives you intelligence about what would otherwise be simple message exchanges, for example.

Start with the Investigating with Activities regarding how Activities are defined and how to work with them. Then go on to these topics:

How to Create Activity Queries

There are three basic categories of parameters: Activity, Devices, and Domain. Every query filters the Activities list, Domain artifacts list, and the Devices list, regardless of which list is visible in the Workbench.

You can filter Activities by using the Feature Summaries panel or by typing in query parameters manually:
  1. Select "AML" rather than "Search" from the drop-down menu at the upper right.
  2. Click in the text field at the top of the window.
  3. To use Feature Summaries:
    1. Select Feature Summaries from the Model Builders category at right. You will see a long list of summaries grouped by category: Device Summaries, Domain Summaries, and Activity Summaries.
    2. You can scroll the list, collapse or expand each category, and search the summaries.
    3. When you move the pointer over a summary, you will see a tooltip with details of that summary. Click on a summary of interest, for example Device Type.
    4. In this way, you can explore the hierarchy of parameters. When you find one of interest, click on it to select it. You can select more than one.
    5. Add it to your query by clicking the + Add to Query button at lower right. This will place your query in the text field at the top of the window.
    6. Continue in this manner as desired.
    7. Proceed to step 5.
  4. Instead of using the GUI and Feature Summaries, you can type your query into the text field at the top, once you have selected AML from the pop-up list.
  5. When you are ready, click the round Search button to submit your query.
Note: You can combine multiple parameters in the same category with && (AND) or || (OR). You can only combine parameters from different categories with && (AND). For example, this is valid:
device.type == "Apple" && activity.http.response.header.length > 1024
This is not valid:
activity.http.request.method == "HEAD" || device.risk_score > 10
Awake will report an error if the query is invalid.

Time Controls and Activities Queries

Just as with devices, each query filters the entities found within the visible time window. The time controls behave in the same manner as they do in the Devices tab.

For details on timeline filtering, see Searching and Query. To adjust the time window using the GUI, see Time Controls.

Activity Filters

As you start to enter a query, you will see a list of possible parameter matches below the query field. See the Filter Activities illustration. Like the Workbench lists, the Feature Summaries panel is first filtered by your query; the panel only shows parameters the system knows are available given your current timeline and query. The panel gives you a collection of entry points to filtering activities.
Figure 16. Filter Activities

When the values list is long, you can search for a value in the list. For many parameters, scrolling through the data can give you a quick understanding of activity in the system. You can also sort the results by name or count. The default display is sorted by descending count.

Weak Signals

Weak signals are anomalous conditions that may not be malicious. Most commonly they are protocol violations and similar conditions. Here are some steps you can take to investigate them.

  1. To find weak signals in your input stream, use the following query:
    count activity.weak >= 1
    In the activities list, the response shows the protocol violation or other area of concern. The weak signal is flagged with an exclamation point.
    Figure 17. Weak Signal Flag

  2. Hover on the exclamation point to see some details on the weak signal itself. The text associated with the weak signal appears above the exclamation point. This text appears in the parameter activity.weak.text, and can be used in queries.
  3. To examine the known extracted data from the packet, click on the Details link in the first column to open the Activity Profile for this activity. See Work With Activity Details for more.
  4. To search for weak signals, or filter your results, use the Activity filters. You can start with the activity.weak.text shown when you hover over an exclamation point in the activity list, as shown above.
    Figure 18. Weak Signals Filter

Work With Activity Details

To see details for any activity in its own Activity Profile, click the Details link in the first column.

Figure 19. Activity Details

This will show the Extracted Data view in a new Activity Profile pane in the Workbench. Here you will see the detailed Activity Profile, which includes:
  • Extracted Data, sorted by protocol.
    Figure 20. Activity Details: Extracted Data

  • Activity Analysis, with a bar graph showing data volume for each event on a timeline, and the corresponding table of events.
    Figure 21. Activity Details: Activity Analysis

  • Captured Data (if available), showing the payload data in the Request and Response messages.
    Figure 22. Activity Details: Captured Data

    The header information is not repeated in these messages. Instead, this area presents the activity data as the application layer sees it. By default, each message is in ASCII format to make reviewing the payload easier. To switch to hexadecimal (Hex) or UTF-8 formats, click on the corresponding radio button above the data.

    From the Activity ID field you can also export a Packet CAPture (PCAP) file just as you can from the Activities list.

Activity Protocol Templates

In the activities list in the Workbench, the Console displays each activity according to a template that shows key information for the highest-level protocol used by the activity. Each template is designed to show, at a glance, the activity that occurred.

Here are a few examples.

HTTP

activity.http.request.method http.request.uri
{, host:: activity.http.request.host}
{, activity.http.request.body.length bytes of activity.http.request.content_type}
{, referer: activity.http.request.referer}
<- activity.http.response.status_code activity.http.response.status_msg
{, activity.http.response.body.length bytes of activity.http.response.content_type}
{, activity.http.response.location}
Note:
  • The request or response payload length and content type are omitted if the request or response has no payload.
  • If there is a payload but no request or response content type, content type is set to the unspecified type value.
  • Other optional fields based on headers are omitted if the header is not present.

Internet Protocol (IP)

IP-Protocol: activity.ip.protocol
Source: activity.ip.source.packets packets, activity.ip.source.bytes bytes
Destination: activity.ip.destination.packets packets, activity.ip.destination.bytes bytes
Note:The data from packet analysis includes messages for the Internet Protocol (IP), and protocols without tailored templates, such as, unknown protocols. This template is used for both, but the latter is more common.

TCP

Source: activity.tcp.source.packets packets,
activity.tcp.source.bytes payload bytes
Destination: activity.tcp.destination.packets packets,
activity.tcp.destination.bytes bytes
Note: This template is typically used only when there are no further parsed records for an encapsulated protocol associated with the record. It is also used for a higher-level application record without a tailored template.

UDP

Source: activity.udp.source.packets packets,
activity.udp.source.bytes bytes
Destination: activity.udp.destination.packets packets,
activity.udp.destination.bytes bytes
Note: This template is used only when there are no more parsed records for an encapsulated protocol associated with the record. It is also used for a higher level, application record without a tailored template.

DNS-UDP

The DNS-UDP template shows the questions from an exchange's request and reply packets as a pair. The transaction ID of the exchange is used to make the match. There is one pair of request/response lines in the output for each unique question, regardless of whether the question was present in a request or a response.

Single Question, Single Answer

The common, simple case is a request and reply pair where there is a single question that is the same in both the request and the reply, and a single answer in the reply. This has the format:

Query: activity.dns.request.question.type
activity.dns.request.question.name
<- activity.dns.response.answer.type
(activity.dns.response.answer.ip | activity.dns.response.answer.name | activity.dns.response.answer.text)
for activity.dns.response.answer.ttl

Single Question, Multiple Answers

Another common occurrence is a request and reply pair where there is a single question that is the same in both the request and the reply but the reply has multiple answers. These have the format:

Query:activity.dns.request.question.type activity.dns.request.question.name
activity.dns.response.answer.type (activity.dns.response.answer.name | activity.dns.response.answer.ip | activity.dns.response.answer.text)
for activity.dns.response.answer.ttl
(, activity.dns.response.answer.type (activity.dns.response.answer.name | activity.dns.response.answer.ip | activity.dns.response.answer.text)
for activity.dns.response.answer.ttl) ... (+n more responses)

The n refers to the number of answers found within the response but not shown. Open the Details pane to see the full list.

Question Without a Reply

A query without a reply has the format:

Query: activity.dns.request.question.type activity.dns.request.question.name 
<- No response observed 

Reply Without a Request

Less commonly, a reply without a request has the format:

No request observed, question from response:
activity.dns.response.question.type activity.dns.response.question.name
<- activity.dns.response.answer.type (activity.dns.response.answer.name
| activity.dns.response.answer.ip | activity.dns.response.answer.text) for
activity.dns.response.answer.ttl(,
activity.dns.response.answer.type (activity.dns.response.answer.name
| activity.dns.response.answer.ip | activity.dns.response.answer.text) for
activity.dns.response.answer.ttl)

Reply and Request Questions Do Not Match

A single request/reply exchange can also have a question in the reply that does not match the question in the request. This is formatted as two separate request/reply pairs of lines with the reply question listed after the request question.

Query: activity.dns.request.question.typeactivity.dns.request.question.name
<- No response observed
No request observed, question from response:
activity.dns.response.question.typeactivity.dns.response.question.name<-
activity.dns.response.answer.type (activity.dns.response.answer.name|
activity.dns.response.answer.ip | activity.dns.response.answer.text) for
activity.dns.response.answer.ttl(, activity.dns.response.answer.type) 
(activity.dns.response.answer.name | activity.dns.response.answer.ip|
activity.dns.response.answer.text) for activity.dns.response.answer.ttl) 

Error Response Code

It is possible that the response to a query contains an error response code rather than success. This has the format:

Query: activity.dns.request.question.typeactivity.dns.request.question.name
<- error: activity.dns.response.code
Note: If there are multiple questions with error response codes, they generate multiple sets of output in this format.

ICMP

There are multiple templates for ICMP, one for each report type generated by packet analysis.

Ping

Ping Request: activity.icmp.ping.request.packets packets, 
activity.icmp.ping.request.bytes bytes
<- Reply: activity.icmp.ping.reply.packets packets,
activity.icmp.ping.reply.bytes bytes

Time Exceeded

activity.icmp.ttl_exceeded.reason exceeded on path from 
activity.icmp.source.ip to activity.icmp.destination.ip 
observed activity.icmp.ttl_exceeded.reason.count times 
Note:
  • The activity.icmp.ttl_exceeded.reason is either time-to-live or fragment-reassembly time based on the TimeExceeded reason field.
  • The source used with this template is the router or end-system that generated the ICMP message.

Path Redirect

Path redirect from activity.icmp.source.ip to 
activity.icmp.destination.ip due to
activity.icmp.path_redirect.reason observed
activity.icmp.path_redirect.reason.count times { with next hop(s)
activity.icmp.path_redirect.next_hop ( ,
activity.icmp.path_redirect.next_hop) ..._n_ } ...
Note:
  • The activity.icmp.path_redirect.reason is presented for known reasons and activity.icmp.path_redirect.reason.code for unknown ones.
  • The with-next-hops section is only present if there is at least one next-hop specified.
  • There is one path-redirect-reason line for each reason observed.

Local Network Configuration


Local network configuration
{address mask requests: activity.network_configuration.address_mask.request.packets}
{, } {router solicitation requests: activity.icmp.network_configuration.router_solicit.packets}
{, } {address mask replies: activity.icmp.network_configuration.address_mask.reply.packets}
{, } {router advertisements: activity.icmp.network_configuration.router_advertisement.packets}
Note: Each optional segment presenting a count is only present if the count is non-zero. The comma segments are only present if both of the segments they separate are present.

Other ICMP

Unusual ICMP activity activity.icmp.other.packets observations of type 
activity.icmp.other.type and code activity.icmp.other.code ... 
Note:
  • The type and code are presented as numbers.
  • There is a separate line for each type and code reported in the record.

Back to Analyzing Activities.