AI and Data Scientists: Striking the Right Balance

AI's impact on professionals, businesses, and society as a whole continues to be a subject of active debate. After more than 8 years of doing the heavy lifting of coding myself, I began using AI more regularly over the past two months to help me write code for my data science projects. In this article, I will share how Data Scientists can benefit from using AI and also provide recommendations on how it (AI) can be used in a way that still allows for well-rounded skill development.

What have I appreciated about using AI? 

1) I was able to work faster. Why?

• AI generated code based on prompts I gave it. In most cases, I gave it a code skeleton and asked it to flesh it out to achieve a specific outcome. 

• I spent less time identifying and trying to understand coding errors, and also less time figuring out how to do something I may not have been very familiar with, like how to use a new coding package or an updated version of an existing one. The feedback and explanations AI provided in these scenarios were generally accurate and easy to understand. (P.S. Gone are the days of reading through multiple posts on StackOverflow or some nebulous documentation to solve a coding issue! 🙂 )

2) Spending less time coding allowed me to spend more time interpreting the results, making connections between them, and figuring out how to communicate them in a clear and simple manner - aka Data Storytelling. That said, a potential concern I have with the narrative of being able to work faster with AI - which is true - is that data scientists will be expected to complete projects within unreasonable time frames. For different reasons, Data Storytelling is often not given the due time and effort it deserves, and this narrative (of being able to work faster with AI) could fuel this problem even more. I therefore encourage Data Scientists to use the “extra time” they have to develop a great Data Story and figure out how to communicate it clearly, and not rush to deliver findings. Additionally, manage the expectations of stakeholders before a project commences by allotting ample time in your project execution timeline for developing the Data Story.

How should Data Scientists use AI? 

Especially for current students in training and junior professionals, I recommend that you

1. Attempt to write code and review a dataset on your own without any assistance from AI before relying on it. Despite AI’s impressive ability to generate accurate code, it is not perfect so having some basic experience with coding will allow you to confidently cross-check whether the output AI provides is correct. Additionally, having basic coding experience can help you give more effective prompts to AI to generate your desired code. 

   
   

2. As much as possible, interpret the results and make connections between them yourself. Although AI is clearly faster than the human brain at performing a number of tasks, I personally believe the human brain has a stronger ability to identify and make intricate connections between results, while tying it to the business objectives; remember, AI is not perfect. Lastly, not doing a majority of results interpretation and establishing connections yourself will likely make it harder to effectively communicate insights and address stakeholder questions.

P.S. Recently, ChatGPT was not accessible for a day and there was a running half-joke that many people were struggling to do their work.😅  With basic coding skills and the ability to interpret results, you will still be able to work in the absence of AI or when AI does not give you the right answer. 🙂

Conclusion

AI is a reality that we have to embrace as Data Scientists and it will only continue to impact the profession more and more. Personally, what I grapple with is how Data Scientists - especially current students in training and junior professionals - can leverage AI in such a way that it still allows for holistic skill development. 

Additionally, given AI’s impressive ability to write code, I believe the most sought-after Data Scientists in today’s world will be those who can effectively collaborate with business stakeholders throughout the life cycle of a data science project, and that includes impactful Data Storytelling - the ability to connect the dots between seemingly disjointed nuanced data insights and communicate them in a way that is easy to understand.

To all aspiring and professional Data Scientists

• In what ways do you use AI to do your data science work? 

• How has AI been beneficial to you? 

• Are there ways in which you believe Data Scientists - especially current students in training - should avoid using AI and why? 

• What skill sets do you think will become most critical as AI continues to evolve?

Remember to share the article with your network if you found it helpful! 🙂