Code Insights Pings

Code Insights pings allow us to quantitatively measure the usage and success of Code Insights. This page is a source of truth for detailed explanations, statuses, and implementations of our pings.

We keep this docs page up to date because pings are a vital component of our product knowledge and prioritization process, and a broken or incorrect ping impacts 3-5 months of data (because that’s how long a fix takes to propagate).

Metrics

Additions count, edits count, and removals count

Intended purpose: To track how many times customers have created, edited, and removed insights, by week.

Functional implementation: The current implementation of Code Insights stores insight configurations in settings files. This ping works by diffing settings files if users edit their settings in the setting editor at the setting edit page. Also, we track insight creating/editing/deleting events in the creation UI form and insight context menu component with standard telemetry service calls.

Other considerations: This is an “imperfect” ping because not all additions + removals directly translate to a new insight or a deleted insight, due to the complications with using settings files as a source of truth. We’ll be fixing this when we migrate to a backend database. Note also we’re using this as a “total insights” metric for the same imperfect reason (additions - removals = total created) and when we migrate to the backend database we should build an additional separate ping that is just “total insights existing on the instance” per week.

Hovers count

Intended purpose: To track how many times users hover over a datapoint to see the tooltip on the graph, or “dig in” to the information.

Functional implementation: This ping works by firing an event on the client when a user hovers over a datapoint on a code insight.

Other considerations: N/A

UI customizations count

Intended purpose: To track how many times users resize the insight graphs.

Functional implementation: This ping works by firing an event on the client when a user resizes a Code Insights graph on the page.

Other considerations: N/A

Data point clicks count

Intended purpose: To track how many times users click a datapoint to get to a diff search.

Functional implementation: This ping works by firing an event on the client when a user clicks an individual data point of an insight graph, which takes them to a diff search.

Other considerations: Because this functionality does not yet exist for backend insights, it only tracks clicks on frontend insights.

Page views count

Intended purpose: To track how many times users view insights pages.

Functional implementation: This ping works by firing an event on the client when a user views any /insights page, whether it’s creating or viewing insights.

Other considerations: As we add new insights pages it’s important to make sure we’re adding pages to this counter.

  • Aggregation: By week
  • Event Code: InsightsPageView
  • PRs: #17805
  • Version Added: 3.25
  • Version(s) broken: 3.25-3.26 (not weekly)(fix PR), 3.30 (broken when switching to dashboard pages, didn’t track dashboard views)(fix PR)

Unique page views count

Intended purpose: To track how many unique users are viewing insights pages each week.

Functional implementation: This ping works by firing an event on the client when a unique user views any /insights page for the first time that week, whether it’s creating or viewing insights.

Other considerations: As we add new insights pages it’s important to make sure we’re adding pages to this counter.

  • Aggregation: By week
  • Event Code: InsightsUniquePageView
  • PRs: #17805
  • Version Added: 3.25
  • Version(s) broken: 3.25-3.26 (not weekly)(fix PR), 3.30 (broken when switching to dashboard pages, didn’t track dashboard views)(fix PR)

Org-visible insights count (Total)

Intended purpose: To track how many insights are visible by more than just the creator of the insight.

Functional implementation: we pull this out of the organization settings files (but this will change after the migration out of settings files).

Other considerations: We should migrate this ping to use the new database after we finish the settings migration.

  • Aggregation: total time, by insight type
  • Event Code: InsightOrgVisible
  • PRs: #21671
  • Version Added: 3.29
  • Version(s) broken: 3.31+ (doesn’t handle backend insights)

First time insight creators count

Intended purpose: To track the week and count of the first time a user(s) creates a code insight, of any type, on an instance. The sum of first time insight creators count over all time is equal to the total number of unique creators who have made an insight.

Functional implementation: This metric queries the insight table for new addition events, then filters by unique IDs that appeared for the first time that week.

Other considerations: This is broken and needs to be migrated when we migrate Insight Addition event.

  • Aggregation: By week
  • Event Code: WeeklyFirstTimeInsightCreators
  • Version Added: 3.25
  • Version(s) broken: 3.31+ (doesn’t handle backend insights, other bugs)

Total count of insights grouped by step size (days)

Intended purpose: To track the x-axis (time window) set by users on frontend insights, to help prioritize features related to setting time windows.

Functional implementation: this metric runs on the backend over all the insights gathered from the settings files.

Other considerations: This ping should be updated when we update the ability to set a custom time window on the backend.

  • Aggregation: total
  • Event Code: GetTimeStepCount
  • Version added: 3.29
  • Version(s) broken: 3.31+

Code Insights View/Click Creation Funnels

Intended purpose: These pings allow us to both understand how the view/click/view/click conversion funnel works for the creation flows of all existing types of insights, as well as smell-check other pings. The reason we use both “view” and “button clicks” in this funnel is that it’s possible to view a page without “funneling through” via the prior page’s CTA (for example: you can reach the creation/edit screen by the “edit” button, which does not involve logging a click on the “create search insight” button).

Functional implementation: These events fire on the frontend when the user takes the below actions.

Other considerations: