Configuring Search Hub
Search Hub governance relies on configurable rules and automation actions that determine how searches are evaluated, flagged, and remediated. Configuration is role-aware and depends on Atlas RBAC:
- Atlas Viewers: Can see search governance results but cannot make changes.
- Atlas Creators: Can create, edit, and test rules in the governance console.
- Atlas Admins: Can configure the system, enable/disable rules, and apply automation through the configuration modal.
Creating and Editing Rules
Atlas Creators and Admins can add or modify rules through the console.
- Rule Name: A descriptive title of the governance rule.
- Description: A description of the Rule. A good place to put recommended fix procedure.
- Condition: The criteria that must be met (e.g., searches running too frequently, searches producing excessive results). See Condition Examples below.
Adding or modifying a rule will prompt the user to run the Governance search early to update the findings early.
Condition Description
Atlas Creators and Admins can leverage these fields when creating a rule.
- savedsearch_name: Name of search.
- savedsearch_app: App of search.
- savedsearch_owner: Splunk owner of Search.
- generating_command: The raw text of the first line of the Splunk query. This is useful for identifying searches with wildcards.
- earliest_time: Earliest time configuration for search. 0 is for all time.
- latest_time: Relative latest time configuration of the search. For example, "-1d@d" would be start of previous day.
- timerange_is_inline: Returns 1 if there is a time range denoted in the SPL of the search, 0 if not.
- window_minutes: Number of minutes between earliest and latest time.
- cron_schedule: String of the Cron schedule.
- interval_min: Number of minutes in between configured scheduled runes. For instance, if a search is scheduled to run every 5 minutes, this will be 5.
- interval_max: Max number of minutes between scheduled searches in time range. Useful if searches don't have set pattern schedules.
- misconfig_minutes: Difference between interval and time range window. Positive is when window is larger than interval (over searching), negative if there is a coverage gap.
- avg_runtime: Average runtime of search in seconds.
- avg_latency: Average latency of search in seconds.
- skip_rate: Skip rate of search in percentage (1 would be 100% skip rate).
- skip_count: Skip count of search (3 would be the search skipped 3 times).
- run_count: Number of times search runs last 24 hours.
- avg_svcs: Average SVC consumption last 24 hours (Splunk Cloud only).
- tot_svcs: Total SVC consumed last 24 hours (Splunk Cloud only).
Conditional Logic
The fields can leverage Splunk conditional logic compared to a 'Where' command. So usage of >, >=, like(), and so on can help with creating useful rules.
Configuration Modal (Admins Only)
Atlas Admins have additional configuration access through the Configuration Modal:
Enabling Search Governance
- Enable/Disable Rules: Activate or deactivate rules globally.
- Target Environments: A selection of environments that these rules will apply against. These need either be the local host or previously configured Atlas Targets.
- Admin Contact: In the event of searches not having an owner, this email will be notified instead. Users will also be directed to contact this email for assistance.
Automated Remediation Actions
- Automated User Alert: When activated, Splunk will send an email to Splunk users with searches breeching rules, notifying them of what searches are failing what rules.
- Escalation Alert: When activated, Splunk will send an email daily to the owner of searches that have been under Warning for the inputted amount of days. This warning will say that Splunk Admins may edit or disable the search.
- Auto Disable: When activate, Splunk will auto disable any email that has been Escalated for the inputted amount of days. Users will be warned of this in the notifying emails.
Best Practices
- Start with Warnings: Configure rules to warn before enabling escalations or auto-remediation.
- Notify Owners Early: Ensure owners are contacted before disabling their searches.
- Audit Regularly: Use Action Logs to review what governance actions have taken place.
- Align with Limits: Configure rules to reflect Splunk system and license limits for maximum effectiveness.