Using Atlas Assessment (Classic)
Upon opening the Atlas Assessment app, it immediately starts the investigation searches and checks to assess your Splunk environment. You should give these searches adequate time to completely analyze and calculate the current status of your environment - the larger your environment, the more data it is ingesting, and the more time will be required to complete the assessment.
When the All Recommendations section at the bottom of the dashboard finishes populating, all the Key Performance Indicators (KPIs) have loaded and the dashboard is ready for consumption.
Note that below the KPIs for the Data Sprawl, Data Awareness, and Usability Gap sections section are some questions related to the section topic - the answers to these questions may change the findings and recommendations.
Ensure you are running this dashboard as a role with Splunk admin permissions for the best results
Atlas Assessment Structure
The Assessment is separated into sections by headers, with each header correlating to an Atlas Theme. Each section contains the following segments:
Help Text explaining what the Atlas Theme is focused on
Questions related to Splunk environment processes and goals
KPIs found by analyzing the Splunk environment
Findings generated by analyzing question responses and the KPIs
Examples of Atlas Assessment Sections
Definitions of Atlas Assessment Sections
Theme 1: Data Sprawl
Data Sprawl in a Splunk environment refers to what happens when Splunk admins begin to lose visibility into what data is being ingested into Splunk, why it is being ingested, and if or how it is being used. Without proper oversight, Splunk environments can become hamstrung by an excessive imbalance of data ingest versus data utilization. This means you could be ingesting and storing data that is not providing a valuable return on your investment in Splunk. The worst impacts of Data Sprawl include unnecessary, excessive Splunk license costs, longer triage times when issues arise (more data to sort through), and mounting technical debt that Splunk admins and management will have to deal with every day in terms of higher Splunk resource and storage costs and associated administration overhead.
KPI Overview
Users have the ability to select the Utilization Threshold for detecting data sprawl. Atlas Assessment measures the amount of ad-hoc, dashboard, and alert searches that are executed over the selected time period; relatively unused data sets are flagged as Underutilized. Review the "What is Data Sprawl and Underutilized Data?" help text for guidance on the different threshold options.
The KPIs for Data Sprawl are:
Underutilized Datasets / Total Datasets: Number of Index - Source Type pairs that fail to meet the Utilization Threshold compared to the total number of Index - Source Type pairs being ingested into Splunk
Underutilized Ingest / Total Ingest: The amount of data being ingested into Splunk during the time range that has not passed the Utilization Threshold, compared to the total ingest volume
Total Ingest / License Entitlement: The amount of data being ingested into Splunk during the time range compared to the Splunk License quota. If the quota is inaccurate, apply a new value in the questions section below the KPIs.
Percent of License Entitlement Underutilized: The percentage of Splunk License that is underutilized through low search activity, or not being used due to low ingest volume
Theme 2: Search Performance
Optimal performance of your Splunk environment depends heavily on properly scheduled searches. To maintain accurate visibility of data on your reports and dashboards, you must ensure that your search scheduler is able to perform in an optimal fashion. A Splunk deployment with poor Scheduled Search Performance leads to Splunk users experiencing long search delays, alerts failing to trigger because the scheduled search for the alert is skipped, and an increased potential for inaccurate data on your reports and dashboards.
KPI Overview
Search Skips: The number of Scheduled Searches that have failed, or 'skipped' in the selected time range
Scheduled Searches with Too Large/Small Time Window: The number of Scheduled Searches that do not have their schedule and time range in sync, which can cause data coverage gaps or wasted resources
Orphaned Searches: Number of Scheduled Searches that lack an owner, causing them to fail to run
Searches Running Every Minute: Number of Scheduled Searches that trigger every minute, potentially causing an adverse impact on on Splunk search performance
Theme 3: Data Awareness
Data Awareness helps you assess the status and health of Splunk forwarders that send data to your Splunk environment. Splunk environments with poor Data Awareness have inadequate visibility and alerting on the stability of their data pipelines, which can result in inaccurate data reporting and critical alert conditions being missed.
KPI Overview
Distinct Forwarder Versions: Number of unique versions of Splunk Forwarders discovered
Forwarders Identified: Number of Unique Forwarders discovered in the Splunk Environment
Forwarders Missing - 15 Min: Number of Forwarders identified as 'missing' in the past 15 minutes
Theme 4: Usability Gap
Splunk is a powerful platform that can achieve significant outcomes when used correctly. We define the Usability Gap as the difference between when your Splunk users first start using Splunk to when they really begin leveraging the platform to achieve real outcomes. The first thing you need to determine is how your users are interacting with the platform today and if the level of activity is what you think it should be. The Atlas Application Suite comes with multiple tools that can help jump start your users so they can realize the most value out of your investment in Splunk quickly.
KPI Overview
Active Users: Number of users found in the specified time range that have logged onto Splunk
Users performing Ad-Hoc Searches: Number of users found in the specified time range that have done an ad-hoc Splunk search through the Splunk Web search UI
Users Hitting Dashboards: Number of users found in the specified time range that have loaded a dashboard
Users Per Admin: Ratio of Splunk users to Splunk admins - a value of 5 indicates five users to one admin
Apps with Version Drift: Number of Splunk apps with different versions across the Splunk Environment
Apps with Local KOs: Number of Splunk apps with local knowledge objects, such as dashboards or scheduled searches
Apps not used in last 7 Days: Number of Apps that no user has loaded in the last 7 days
All Findings
At the bottom of the Assessment is the final section, “All Recommendations”. This section combines the findings of all Atlas Themes and presents it in an exportable format.
All Possible Atlas Assessment Findings
Code | Finding | Recommendation |
---|---|---|
DS01 | This Splunk environment does not have a data request process in place. An integrated process for Splunk users that defines how data onboarding is managed can help avoid Splunk license costs before it becomes an issue. | Atlas's Data Management Element provides Splunk administrators with a Data Request system. This tool allows Splunk users to make requests for data onboarding from within Splunk. Administrators can then manage the request process and also collect metadata about the data being on-boarded. This will create an inventory of the data during the onboarding process that will help to identify key information like the use case that the data is supporting so that you can understand the business impact of your data sources in Splunk. This helps to keep Data Sprawl at bay while also improving user engagement within Splunk. |
DS02 | This Splunk environment has no system in place that can help with tracking data ownership over the dataset_c Splunk source types found in the environment. This can directly lead to Data Sprawl because there is not visibility into who owns the data or who is impacted if the data ingest fails. Typically, over time, the reason that data was brought into Splunk is forgotten and who requested or owns the data is lost. This leads to Splunk admins who are uncertain about the business impact that the data has and who to reach out to when issues arise. | Atlas's Data Management Element provides a Data Inventory feature that allows Splunk administrators to assign ownership to source types in use within their Splunk environment. Data Management will automatically inventory all of your data sources and will let you assign data definitions to each index and source type pair. The data definition includes a data ownership Element that can be used to assign business users to each data source. Maintaining a robust data inventory for your data sources is key to avoiding Data Sprawl. |
DS03 | Data Sprawl is likely impacting this Splunk environment because at least 15% of your Source Types are underutilized. Utilization is measured by the number of searches that are accessing your data. When data is not searched, or lightly searched, it is classified as underutilized. When you have a high amount of underutilized data this could be increasing the technical debt of the environment, while also increasing costs without increasing Splunk outcomes. | Atlas's Data Utilization Element can provide Splunk admins with unprecedented access into how your data is being utilized in your Splunk environment. With Atlas Data Utilization you can see precisely who is using your data, how frequently the data is being accessed and how it is being used. Having the ability to audit your data utilization at this level of detail helps Splunk administrators make informed decisions about their Splunk environment when it comes to things like triaging issues, optimizing their environment, and making Splunk license purchasing decisions. |
DS04 | Underutilized data is consuming DS_SizeUnderUtilizedGB GB of your current data ingest within the time frame that is specified. This is more than 15% of your Splunk license spend which is the threshold this assessment uses to determine if there is data sprawl in this environment. | Audit your Splunk data with Atlas's Data Utilization Element to get visibility into exactly how your data is being utilized. Since you have a high amount of underutilized data in your Splunk environment, you should consider reducing the amount of data you are ingesting or identify ways to increase the output of the Splunk environment to better utilize the data you have in Splunk. Underutilized data is not bad in all cases but can carry high costs and add significant overhead to your Splunk environment. |
DS05 | X% of the Splunk License Cap is not being used, or is filled with underutilized data. This is more than 15%, which is the threshold this assessment uses to determine if there is data sprawl in this environment. | Audit your Splunk data with Atlas's Data Utilization Element to get visibility into exactly what indexes and source types are not being utilized by Splunk, and determine if a lower license is possible for your instance. |
SP01 | This Splunk environment could be experiencing poor Search Performance because SP_skips alerts and scheduled searches were skipped in the selected time range. Skipped searches can lead to missed alerts, inaccurate reporting, and an overall failure of monitoring objectives. | Identifying and fixing skipped searches can be a difficult and time-consuming process for even the most experienced Splunk administrator. Using Atlas's Scheduling Assistant, you can easily identify searches that are being skipped, test scheduler changes, and apply scheduled search changes within a single user interface. Scheduling Assistant is designed to make the process of optimizing your search scheduler easy giving your Splunk administrators hours back in their day so that they can focus on getting outcomes from Splunk rather than fixing it. |
SP02 | There are SP_gaps searches in this Splunk environment that are currently not meeting recommended best practices because their search time ranges do not align with their execution schedule. These misconfigured searches impact Search Performance because they can miss critical events in your Splunk data and waste resources on unnecessarily long running searches. These can quickly contribute to deteriorating performance in a Splunk environment. | You should investigate and identify misconfigured scheduled searches in your Splunk environment to ensure that your searches are not missing data or that you aren't over searching your data putting an unnecessary strain on your Splunk environment. With Atlas's Scheduling Inspector, you can quickly identify these searches. Scheduling Inspector will recommend modifications to your search schedule that will correct misconfigured scheduled searches and allow you to automatically apply the changes directly from the user interface. |
SP03 | There are currently SP_orphan orphaned searches in this Splunk environment. Orphaned searches are scheduled searches that have been deactivated by Splunk when the user account in Splunk that owns the search has been removed. The search scheduler in Splunk cannot run a scheduled report on behalf of a nonexistent owner. This can result in broken dashboards and data collection gaps that negatively impact your Splunk environment. | Orphaned searches should be identified in your Splunk environment and reassigned to an existing Splunk user or deleted altogether. Atlas Scheduling Inspector provides an integrated tool for identifying and resolving orphaned searches within Splunk. You can also automatically delete the search if it is no longer needed. |
SP04 | Search Performance is being negatively affected by SP_badSearches searches that run too often in your environment. These searches can make Splunk slower for users, and cause other searches to skip! | Identify these bad searches in Scheduling Assistant and give them better scheduling so they don't harm your Search Performance. |
DA01 | Based on your responses to the questions above this Splunk environment does not have an alerting system in place for monitoring critical data streams. Having a robust alerting system is paramount for building an appropriate level of data awareness in your Splunk environment. This ensures that you are alerted when there is an issue so that appropriate action can be taken and your Splunk users are not missing critical alerts or data on their dashboards. | You should have alerts in place for when data ingests behave unexpectedly. This allows you to proactively ensure that your Splunk users are not missing data. Atlas's Data Management Element provides a feature called Data Watch. Data Watch is an integrated feature that allows you to quickly put a watch on critical data sources to ensure that you are alerted when there is an issue that could impact your Splunk. In many cases, when this problem occurs, your Splunk users are unaware that a problem even exists. Data Watch can help to ensure that your dashboards are accurate and that your Splunk users are not missing critical alerts. |
DA02 | Based on your responses to the questions above, this Splunk environment might have Enterprise Security installed and you are uncertain that you have all the data you need to realize the value from the platform. Enterprise Security requires a large amount of data to fully utilize all its features. Enterprise Security is a significant investment and can be a critical tool supporting your organization's mission. | To ensure that you are properly populating the data models supporting Enterprise Security and that you have all the important data that you need to get value from the platform you should leverage Atlas's Enterprise Security (ES) Helper Element. ES Helper will evaluate your ES implementation and score how well you are doing for providing ES the data it needs. ES Helper also will provide you with the guidance for which data you should prioritize next and will track your progress as you improve your ES implementation. |
DA03 | There are DA_versionDrift distinct Splunk Forwarder versions found that are sending data to this environment. Having version drift between your forwarder versions may cause you to lose functionality or could present more serious issues when you have forwarders that are too old. This can prevent you from being able to upgrade your Splunk infrastructure leaving you vulnerable to major security issues. | You should ensure that your Splunk forwarders are running the latest version of Splunk. Running older versions of Splunk forwarders leaves your environment vulnerable to functionality loss or security issues. Atlas's Forwarder Awareness Element allows you to quickly identify which Splunk Forwarders are actively sending data to your indexes, which versions of Splunk they are running and if they have SSL enabled or not. |
DA04 | There are DA_FWDCount Forwarders found on this Splunk environment. Having up-to-date visibility into the health of your forwarder population is a critical aspect of Data Awareness. | Using Atlas's Forwarder Awareness Element you can group your forwarders into Forwarder Groups. Forwarder Groups allow you to provide context about your population of forwarders and associate ownership information to each group. This information can then be used to alert owners when there is a forwarder outage. |
DA05 | This Splunk environment could be missing data because there are DA_missingCount forwarders not currently reporting. These missing forwarders can lead to inaccurate dashboard and missing critical alerts. | These missing forwarders should be investigated to ensure that your dashboards and critical alerts aren't missing data. You can utilize Atlas's Forwarder Awareness Element to identify down forwarders and easily see the how your data is impacted by any missing forwarders. You can also setup alerts to ensure that you are properly notified the next time a forwarder goes offline. |
UG01 | Based on your answer to the question above you indicated that your Splunk administration Team could use additional help with things like onboarding data, making searches, creating dashboards and more. | Every Atlas subscription comes the Expertise On Demand (EOD) service. EOD is your own personal task force of Splunk experts that can assist you with achieving outcomes in Splunk. The Atlas EOD team is equipped to help you with not only solving complex Splunk problems but can also be an extension of your Splunk administration Team. They are trained to help customers bring more use cases into Splunk so that you can do more with the platform. |
UG02 | Based on your answer to the question above you indicated that your Splunk administration team could use a tool that can help Splunk users run effective searches without having to learn SPL. | The Atlas platform comes with an Element called Search Library. The Search Library is pre-populated with many optimized and Splunk searches that you and your team can utilize right out of the box. In addition, Splunk administrators and Users can populate the Search Library with their own custom searches so that they can be shared with others across the team. The Search Library is an effective tool for getting your users up and running in Splunk quickly and easily freeing up your Administrators to go focus on more mission critical tasks. |
UG03 | There are only "X" active user(s) in this Splunk environment. This may indicate that you need help with onboarding new users into your Splunk environment or getting more from your Splunk environment. | The Atlas Expertise on Demand service is included with an Atlas subscription and is there to help you do more with your Splunk environment. The Splunk experts on the EOD Team can assist with onboarding new users into Splunk, onboarding new data, or creating a personalized education plan for your Splunk team. |
UG04 | There are "X" active users in this Splunk environment. With more users using Splunk there is an increased risk of a Splunk environment becoming burdened with issues like scheduled search concurrency, poor visibility into what data you have and how your users are using it. | The Atlas platform is designed to help Splunk teams tackle common issues that they encounter as a Splunk environment grows. If not addressed, some of these issues can start to degrade your Splunk performance and slow your adoption efforts because your Splunk administrators are spending more time fighting fires than they are working on growing the platform. The Atlas Expertise on Demand Team is designed to help you take full advantage of all the features of the Atlas platform and keep your Splunk administrators focused on growing Splunk in your environment. |
UG05 | Atlas has detected only "X" users using dashboards in this Splunk environment. This isn't that many and could be a sign that Splunk users and Power Users are not getting the value from Splunk that they want. | The Atlas Expertise on Demand can assist the Splunk admin Team with enabling their Users to fully utilize the Splunk platform and make the most of their investment in the platform. The Atlas platform brings your Splunk users tools like Search Library that can help to give your users access to a library of optimized queries that they can run without having to learn SPL. Splunk administrators can put their own searches in the library so that their users can run the queries that they need to meet their specific needs. Search Library is a great tool for jump starting your users in Splunk. |
UG06 | Atlas has detected users performing ad-hoc Searches on the Splunk Instance! This is enough for activity for these searches to start negatively affecting the Splunk environment. | Splunk administrators should ensure that their Splunk users are adequately equipped for building good, optimized searches. With Atlas's Search Library Element, you can quickly identify frequently executed searches in Splunk's search activity and ensure they are following Splunk's recommended best practices. Frequently executed searches can be quickly added to the Search Library, optimized and made available for users to execute. |
UG07 | Atlas has detected only Users users using the dashboards on this Splunk Instance. This isn't that many and could be a sign that your Splunk users are not having their needs met by the current Splunk Deployment. | Atlas's Expertise on Demand can assist the Splunk administration Team with enabling their Users to fully utilize the Splunk platform and make the most of their investment. |
UG08 | Atlas has detected users using dashboards on this Splunk Instance. This is enough for dashboard searches to start negatively effecting this Splunk environment. | Your Splunk administrators should ensure that the dashboards are using optimized searches and utilizing the correct data to ensure Splunk performance stability. With Atlas, the Expertise on Demand team can assist your Splunk administration team with reviewing your dashboards to ensure they are not negatively impacting your environment and ensure that Splunk's best practices are being followed. |
UG09 | This environment has more than 10 Splunk users per Splunk administrator. Your Splunk user to Administrator ratio is high which could mean that your Splunk administrators are being overtaxed with maintaining the environment and not meeting the needs of your Splunk users. | With Atlas's Expertise on Demand, Splunk administrators will not only be equipped with the proper tooling to make managing Splunk easier but can also leverage the EOD Team to extend their Splunk administration team to get more done and adequately meet the needs of their Splunk users. |
UG10 | Atlas has detected Splunk applications that have differing versions deployed across your Splunk search head and indexer layers. This can be problematic for your Splunk users because it can negatively impact the consistency of data and visualizations in your Splunk environment. | Atlas's App Awareness will quickly identify which Apps have differing versions across your Splunk environment and tell you which search heads and indexers are impacted. This can enable Splunk administrators to quickly identify and remedy Splunk app version drift issues. |
UG11 | Atlas has detected Splunk apps that contain local knowledge objects. Local knowledge objects are created by Splunk users and can increase the complexity of managing your environment if not monitored effectively. | With Atlas's App Awareness, Splunk administrators can identify which Splunk apps have local knowledge objects that were created by Splunk users, and can identify their type, owner, and purpose quickly. |
UG12 | Atlas has identified Splunk apps that have not been used in the past 7 days. That comes out to X % of the Splunk apps installed! This is above 50% and can denote an environment in need of a tune up and audit. | Using Atlas's App Awareness, a Splunk administrator can quickly identify what apps are popular and what apps are no longer being utilized. Admins can further investigate which dashboards on these Splunk applications are providing value to users to help reduce artifact sprawl and increase clarity in their Splunk environment. |