AI-Powered Analytics and the Future of Analytics Professionals

Last year, I wrote an article - AI & Data Scientists: Striking the Right Balance - addressing how aspiring and current analytics professionals can benefit from using AI. I also provided recommendations on how it (AI) can be used in a way that still allows for well-rounded skill development.

In this article, I explore how:

• AI powered analytics is impacting the job security and prospects of analytics professionals

• AI can be used to build domain understanding in analytics projects

How AI-Powered Analytics Is impacting data professionals

Historically, data analysis was typically performed either by writing code in languages such as R or Python, or by using graphical tools such as Excel or Power BI. While these approaches remain dominant, advances in AI-powered analytics tools are making it possible for users to analyse data using natural language.

Large language models such as ChatGPT are one example. Users can upload a dataset and ask questions about it in plain language, without needing to write code. For example, a business stakeholder without a background in analytics could ask, “What is the average income of customers living in Tema?” or “What are the key drivers of customer churn?” and receive insights derived from the uploaded dataset.

Since these Natural Language Processing (NLP) analytics tools eliminate the need for code and give virtually anyone the ability to glean insights from data, some argue that analytics professionals may no longer be needed. While these tools make data analysis more accessible, there is still a need to understand the principles and key steps to be taken for comprehensive data analysis. For example, how should missing values be handled? Should the rows with missing values be removed or should the missing values be filled with the mean? The quality of the analysis often depends on making appropriate decisions at these stages. 

I observed this firsthand while teaching an R for Data Analytics course. A student shared that she attempted to use AI to complete a homework assignment but was unable to obtain satisfactory results. It wasn’t until she made the effort to review and understand the underlying analytical concepts that she was able to successfully complete the assignment. 

This goes to show that the knowledge analytics professionals have gives them an upper hand in using these NLP tools over individuals who don’t have an analytics background. It therefore isn’t surprising that after all the AI-related layoffs, Forbes and other sources are now reporting that “29% of organisations which cut employees due to an AI related reduction had already rehired into the positions they cut” and “55% of executive decision makers who replaced their employees using AI will regret the decision within 18 months.” I therefore believe analytics professionals will ultimately remain relevant for the foreseeable future. 

Additionally, over time, I believe more analytics professionals will also be involved in the development of customised NLP analytics tools. Since datasets have different characteristics and nuances, generalised NLP analytics tools like ChatGPT don’t always analyse a company’s data effectively. Because of this, NLP tools are being built that are tailored to a given company’s datasets. Customised code is written and runs behind the scenes to execute the unique data cleaning and transformation steps required to generate more accurate insights. This is a new area of analytics work that I suspect will become more prevalent. 

Using AI to build domain understanding in analytics projects

In my article, AI & Data Scientists: Striking the Right Balance, I discussed how AI is helpful for generating, understanding, and debugging code. As I have continued using it consistently over the past year, I have also found it particularly useful in acquiring industry knowledge to work on different projects.

I was recently working on a loan default project. There were several variables in the dataset including:

• the customer’s income 

• the amount the customer still owed on the loan

• The customer’s debt to income ratio

My initial assumption was that the “debt-to-income ratio” was simply the “amount the customer still owed on the loan” divided by the “customer's income”. However, after checking on ChatGPT, I learnt that the ratio takes into account the customer's total debt obligations (including mortgages, auto loans, credit card balances etc) and not just the specific loan in question. This helped me avoid an incorrect assumption about the data and better understand the meaning of the variable. Additionally, ChatGPT was helpful in recommending metrics and insights to explore, while also explaining why they would be meaningful.

This is one of several examples where AI helped me quickly acquire the knowledge needed to work on a project in an unfamiliar domain. Ultimately, AI has made it easier for analytics professionals to work across different industries and communicate in industry-specific terminology.

Conclusion

It continues to be a journey of learning and discovery as I explore the use of AI in my work and its impact on the field. I am by no means aware of every AI-powered analytics tool, but based on what I have seen so far, I believe there is still a strong need for human analytics professionals.