I interviewed 46 senior leaders about challenges on unstructured data. This is what I learned.

Sweeping unstructured data under the rug.

Over the last few weeks I’ve been on the phone with 46 senior leaders from various sectors — oil & gas, insurance, retail, charities, banking and even agriculture, trying to understand the biggest challenges around unstructured data. Once again, A HUGE THANK YOU 🙌🏼🙌🏼🙌🏼 to those who gave me your most valuable asset — time and shared your insights. The most repeated challenges I heard were:

• Visibility: Orgs. don’t know where much of their unstructured data is held or what it contains or when data is copied from structured sources to unstructured formats, makes it difficult to know how big the issue is and whether new measures need to be introduced. Computing power is becoming cheaper and cheaper so they can afford storing it (you never know when will it be used). However, it brings a lot of other challenges like…
• Access — Everybody thinks a “single source of truth” would solve much of the problem but it seems not to be on priority agenda. When it comes to privacy & policy / compliance, accessing and sharing contracts and documents is usually done via Sharepoint. You can’t really find anything on it (keyword search is not ideal if you don’t know EXACTLY what you’re looking for). . They believe Sharepoint is either a) “good enough” — employees eventually find what they want b) employees don’t even bother looking as they believe “ask for forgiveness” strategy is better. Either way, it seems not to be the most burning issue.
• Privacy: This is on everyone’s agenda but apart from GDPR and doing absolute minimum (because some companies are forced by regulators — but not always. Things are e.g. more relaxed in Emirates or Asia) If an organisation isn’t able to find data in unstructured sources, it risks not being able to provide somebody with the information they have requested within the compliant timeframe. Also, organisations that share data with partners might pass on unstructured data to them and no longer have access to it themselves. The most obvious area where it matters is legal. A legal issue could arise with a customer or partner that relates to terms agreed over email at the beginning of a relationship. With data held in unstructured sources, it is difficult to search for and retrieve the information required as evidence.
• Financial — This seems more of an issue in charity sector. Organisations that collect data without managing it in structured sources risk needing to pay for more space and technology to store the data, rather than solving the core problem. — The least of a concern seems to be security around unstructured data (but I may have not gathered enough data or/and I’m biased since i wasn’t focusing on that aspect). Companies rely on sturdiness of AWS / Azure to help them with that.

We at Untrite breathe technology (especially Machine Learning and Natural Language Processing) but often struggle to understand the most burning, urgent problems we should be focusing all our efforts on.
🤔 Please share your views or/and someone who may have an interesting perspective on that. 🤔

Thank you!