Key points:
- Institutions can transform hidden data into actionable insights
- The EDUCAUSE Top 10: Rebuilding trust in higher ed
- Data-informed decision-making in education: A comprehensive approach
- For more news on dark data, visit eSN’s Campus Leadership hub
In the world of technology and information, dark data refers to the information organizations collect but do not use. According to a 2019 Gartner.com study, over 80 percent of organizational data goes unused, sitting idly in servers as “dark data” while companies struggle to make informed decisions. This data exists, but remains hidden and unused.
According to Paramita Ghosh, dark data is the “information that has been collected but not actively used or analyzed, leaving its potential benefits to remain speculative.” Such data remains dark for a variety of reasons. It might be as simple as a lack of staffing or proper analytical tools, the inability or time to organize the data into usable formats, the siloed nature of some of the data sets, or even a lack of staff data literacy. For example, a school might track how many students log into an online platform, but it does not analyze what times students are most active or how long they spend on the site. Another example is a case where the university’s data team gathers data based upon the way a college’s departments were previously organized. When the new departments ask for data, the data does not align and is not able to be used effectively. The data is still there, but it is not in a useful format.
The concerning thing about dark data is that it may have value if someone took the time to review it. For instance, a university bookstore might have customer feedback forms that no one ever reads. If they did, they might learn how to improve their products. If no one ever uses it, that data just sits there, wasted. So, dark data is about the information organizations have collected but do not pay attention to. In some cases, the organization might not be fully aware that the data has been collected. If this data is found and transformed into a usable format–for instance, while browsing related data sets–it might help decision-makers produce better decisions.
Most organizations collect a wide range of data that is not utilized effectively, often due to resource constraints or lack of awareness. This includes everything from security camera footage that is only reviewed in emergencies to website analytics that track user behavior patterns but are never analyzed in depth. Technology help desk call records, logs from servers and learning management systems (LMS), and geolocation data are all additional sources of data. Another example is when an advisor reviews LMS data to identify that the students who log into an online course during a preview period at the beginning of the semester have higher class grades than those who do not log in during the preview period. Then, the advisor shares that information with his advisees, so they know to make more effective use of the preview period. The LMS data was collected, but it was not being analyzed to create useful information. It was another example of how dark data can be proved useful.
The challenge is not just about having the data; it is about having the right tools, expertise, and time to extract meaningful insights from it. Another potential source of dark data to be harvested is from social media. While universities may well do an excellent job of using social media data effectively from university level accounts, college and individual program-level accounts are often overlooked. Comments, direct messages, and engagement metrics from college or program-level social media platforms can potentially help gather more targeted insights to assist in recruiting and alumni engagement.
Imagine a university department managing its own social media accounts to promote programs and events. Over the past year, the department’s Instagram account received hundreds of comments and direct messages from prospective students. These interactions often contained questions about application deadlines, program details, and internship opportunities. However, the department only replied to queries as they came in and did not analyze these interactions further.
If the department utilized this social media “dark data,” it could identify trends, such as the most frequently asked questions, the timing of heightened engagement (e.g., during application cycles), or the types of posts generating the most interest. For instance, it might notice that posts featuring alumni success stories receive higher engagement and prompt more inquiries about career outcomes.
At a mid-sized university, the Office of Institutional Research collected vast amounts of data from various sources, LMSs, student surveys, and even building access logs. However, much of this data remained unused, primarily due to a lack of tools and expertise to analyze it effectively. For instance, while the LMS tracked student logins, activity patterns, and assignment submissions, faculty only reviewed grades, leaving valuable insights about student engagement and behavior untouched.
Recognizing dark data as a missed opportunity, the university implemented several solutions. First, it introduced data dashboards that visualized key metrics in real time, making it easier for faculty and administrators to interpret trends. Next, it hosted workshops to improve data literacy across departments, teaching staff how to analyze and act on data insights. Finally, it adopted a more collaborative approach to data sharing, breaking down silos between offices to allow for integrated datasets.
These efforts paid off when faculty discovered a correlation between students who participated in early-term LMS preview periods and higher overall grades. Acting on this insight, advisors began encouraging students to log into courses during the preview period to familiarize themselves with expectations and resources. This small adjustment, informed by what was once dark data, led to measurable improvements in student retention and performance. By addressing the barriers of tools, training, and collaboration, the university unlocked the potential of its dark data to drive meaningful change.
Even without any immediate benefit, such dark data still uses resources and storage space. Therefore, the efficacy and efficiency of a data-informed organization’s data strategies depend on how well it uses this data. To effectively use dark data, institutional data managers must ensure access to all data sets so everyone can share and work together. Such data democratization can assist in creation of a culture in which all employees utilize data to guide all their decisions, rather than only using it when higher-ups ask them to. Understanding and leveraging these sources can help leadership at all levels of the university optimize operations, improve decision-making, and enhance student experiences.
Dark data holds untapped potential for universities and organizations to optimize operations and enhance decision-making. By democratizing access, improving data literacy, and investing in analytics tools, institutions can transform hidden data into actionable insights. Leaders must act now to bring dark data into the light, ensuring no opportunity is wasted in the pursuit of better outcomes.
Dr. Steve Baule is a faculty member at Winona State University (WSU), where he teaches in the Leadership Education Department. Prior to joining WSU, Baule spent 28 years in K-12 school systems in Illinois, Indiana, and Iowa, and two years teaching in the University of Wisconsin System.
Dr. Rhea Walker is a professor and chair of the Education Studies Department at Winona State University. She holds a Ph.D. from Iowa State University.
- The pursuit of post-crisis leadership in higher education - February 7, 2025
- When adopting AI, colleges shouldn’t override human connections - February 6, 2025
- Trump deportation threats weigh on groups offering FAFSA help - February 5, 2025