Overview
Using the Trace Analytics dashboard you can do a system-wide search on Endpoint traces and spans. Trace Analytics provides various filter options for analysis of traces and spans. Trace Analytics provides you with the option of visualizing the trace data based on various metrics. The Traceable platform receives raw span data from its agent. Traceable's business logic creates the trace and the Endpoint trace (the sub-segment of the trace) from these spans.
The Trace Analytics page also gives you the capability to view raw span data. Spans are the building blocks for a trace. Traceable captures all the request and response data, for example, request headers, body, cookies, and so on are captured as individual span attributes. These spans are put together by Traceable to build a trace. Trace Analytics provides both, the Endpoint Traces and Spans tab to search through.
Filtering Traces
Traceable offers a flexible way to query your traces combining predefined attributes that are specified in a trace (see table), as well as a way to query request/response data like headers, cookies and bodies. Additionally you can query using regex like operations to get very granular.
Operators
Operators are characters that would be used to control what results are returned for a given query, the table below describes the different operators available and what they do.
|
= |
equals |
| != | not equals |
| < | less than |
| <= | less than or equal to |
| > | greater than |
| >= | greater than or equal to |
| ~ | contains string |
| IN | Value is present in multi-valued attribute |
| NOT_IN | Value is not present in multi-valued attribute |
| CONTAINS_KEY | exact match on key in key value pair (request/response headers) |
| CONTAINS_KEY_LIKE | contains string within key for a given key value pair |
Note: Every attribute may not support all operators i.e Protocol would not allow the use of < | <= | > | >= as it would never have a numerical value.
Querying using multiple values
Most fields within Traceable support the use of regex like operations as per the Golang regex library (RE2) - a common use case would be to query a single attribute against multiple values. This can be done like so:
Query Attributes
The following tables outline the attributes that can be used to query traces:
Complex Attributes - These attribute would require you to add additional information to the attribute itself using "dot notation" i.e "attribute.user.input = filter_string"
| Attribute | example | notes |
|---|---|---|
| Request Header |
Request Header.content-type ~ application/json |
To query on a specific header append the name after a . Request Header.<header_name> |
| Response Header |
Response Header.content-encoding ~ gzip |
To query on a specific header append the name after a . Response Header.<header_name> |
| tags (attributes) |
tags.servicename ~ auth |
To query on a given attribute append the attribute name after a . tags.<attribute.name> |
Note: attributes in the context of the tags attribute are shown in the attribute tab of a given trace
Simple Attributes - these are attributes that can be used in a query without having to add information to the attribute itself unlike the examples above i.e "attribute = filter_string"
| Attribute | Definition |
|---|---|
| API exit call | Number of outbound calls made by the API Endpoint |
| Calls | Number of calls to the API Endpoint |
| Duration | Time spent within the boundary of a service that contains the API Endpoint, that is the time between the entry span into service and the exit span out from the service. |
| End time | End time of the segment of the trace |
| Endpoint ID | The ID of the API Endpoint this trace segment belongs to |
| Endpoint Name | Name of the API Endpoint this trace segment belongs to |
| Endpoint Trace ID | The span ID corresponding to the entry span that is the starting span of the Endpoint trace segment. |
| Environment | All the data visible is scoped under the selected environment. But if you select AllEnvironments then this shows the environment corresponding to the trace segment. |
| Error count | Count of error in the given API trace |
| HTTP Method | HTTP Methods, for example, GET/POST/PUT/DELETE and so on. |
| IP Address | Client's IP address |
| Is Endpoint PII | True if the Endpoint corresponding to the trace segment has any sensitive information/attributes. |
| Protocol | HTTP or GRPC |
| Request Body | Contains the API request for the given API |
| Request Content-Type | Content-type request header |
| Service ID | ID of the Service that owns the API Endpoint trace this segment belongs to |
| Service Name | Name of the Service that owns the API Endpoint this trace segment belongs to |
| Session ID | User session ID in which the current operation (that resulted in the trace) was performed |
| Start time | Start time of the segment of the trace |
| Trace ID | The trace ID of the trace that this segment is part of |
| User ID | Username or client_id extracted from the access token or Authorization header in the inbound request from outside the system. |
| User Role | The actual or assumed role the client is making the request under extracted from token or application security context. |
| User Scope | Scopes or granted authorities the client currently possesses extracted from token or application security context. The value would come from the scope associated with an OAuth 2.0 Access Token or an attribute value in a SAML 2.0 Assertion. |
Example Queries
| Traces for a specific session containing PII and returned 200 |
Session ID = <session_id> Is Endpoint PII = true Status Code = 200 |
| Traces for authenticated users on a given endpoint |
User ID != null Endpoint Name = POST /cart/checkout |
Visualizations
You can also create custom visuals to compare against various metrics. One or more metrics can be added for the x and y-axis. The following screenshot gives an example of how you can use Trace visualization.
In the above example, you are viewing the Average Duration metric that is grouped (in a group of 5) by Endpoint Name over a time interval of 1-min. When you hover the mouse over the columns, a pop-up window shows the call duration for all five Endpoints. The visualization shows us that at 7:46 PM, the average call duration forPOST/app/learnt/QKPmJNWoGI endpoint is 3ms. In a similar way, you can create a number of visualizations to compare various attributes.
For more information on the Trace explorer see: https://docs.traceable.ai/docs/traces