British data scientist survey: External data key to success, but poor quality remains a challenge

However, the survey also found that poor data quality is a major challenge for data scientists.

The report, “The Importance and Impact of Using External Data to make Critical Decisions”, by Doorda, a British analytics-ready data provider, is based on a survey of 300 data scientists. The survey found that 92% of respondents said that their models had been significantly improved at some point by adding external data.

However, the survey also found that 40% of respondents had used data that later proved untrustworthy or unavailable. The main challenges that respondents faced when integrating external data were poor data quality (34%), incomplete data (33%), and incompatible data (31%).

“Insurtech companies hoping to compete successfully must look for improvements in the quality of the data they discover and integrate into their analysis to drive innovation,” said Clifford McDowell, founder and CEO of Doorda.

The survey also found that external data is essential for the data scientists surveyed, with the majority (70%) of their models containing at least 20% of externally sourced data. 78% of respondents said that adding new external data had, at some point, exceeded their expectations.

The survey of 300 people, conducted in June 2023 by OnePoll, included those responsible for data science, data analytics and predictive models. They worked in the insurtech, insurance and pensions, financial services, fintech, and professional services sectors.