Dataiku executive highlights the need to use AI responsibly

Sid Bhatia, Regional Director – Middle East, Dataiku, penned an exclusive op-ed for January’s edition of CNME that examines key data science trends that he believes will come to fruition in 2021- which includes the growing need for enterprises to apply AI more responsibly.

In 2020, data science, machine learning, and AI have emerged as critical organisational assets for handling large-scale change with less friction.

Here is a teaser of a few of our data science trends to look out for in 2021 to ensure your organisation is taking a holistic approach (think agile, responsible, and collaborative) to its data initiatives:

MLOps Will Become Even More Critical

Last year, we predicted that the year 2020 will almost certainly be one where MLOps processes are formalised and staffed as well as bolstered up with tools that make the job smoother and more manageable — little did we know at the time how true this would actually become.

Organisations will take their MLOps foundations and go a step further to implement detailed processes and requirements around drift monitoring using MLOps. Input drift is based on the principle that a model is only going to predict accurately if the data it was trained on is an accurate reflection of the real world.

If a comparison of recent requests to a deployed model against the training data shows distinct differences, there is a high likelihood that the model performance is compromised.

In 2020, the significant drift observed was a result of the global health crisis. As a result, the new year is bound to include organisations using MLOps to put more structure in place around drift monitoring so that models can be more agile and accurate.

And organisations won’t stop there. Aside from using MLOps for the short-term to address model drift during events during a crisis, teams will also likely look to implement MLOps practices for the long term in an effort to more effectively scale their machine learning efforts.

Teams Will Need to Infuse Agility Amidst a Post-Pandemic Environment:

From a people perspective, the year 2020 more or less normalised remote and hybrid working styles, making collaboration even more critical for agility and efficiency. It’s also going to take time for organisations to navigate their unique market and understand the ever-evolving dynamics in a post-pandemic setting.

According to Gartner, the theme of resilient delivery “isn’t about ‘bouncing back’ — it’s about having the ability to nimbly adapt or pivot in a dynamic business or IT environment. The theme’s underlying assumption is that volatility exists, so it’s vital to have the skills, capabilities, techniques, operational processes and systems to constantly adapt to changing patterns”.

In 2021, the use of AI for sustained resilience will be underscored, particularly with regard to empowering every team and employee to work with data to improve their business output. These challenges we observed in 2020 will remain in 2021 for teams that don’t have a collaborative data science platform:

Access to systems: Whether accessing the various data sources or the computational capabilities, doing so in a remote setting can be challenging.

Collaboration within teams: Without the physical in-office proximity, individuals can become siloed in the execution of their data projects.

Collaboration across teams: Data projects require buy-in and validation from business teams and also require data engineering and other teams to help with operationalisation.

Reuse over time: Capitalising on past projects is key to maintaining productivity and reducing duplicate work. The lack of in-person discussions can limit this ability.

Organisations Will Go From “What Is Responsible AI?” to “How Can We Implement Responsible AI?”

Up until now, a lot of the conversations around the topic of Responsible AI have been “We haven’t thought about this yet” or “How can we think about it and acknowledge the harms and impacts that AI can have on the world?”.

Teams might be determining how Responsible AI differs across job functions (data scientist vs. an analyst, for example), agreeing on and establishing a framework for their organisation’s ethical rules, and putting checklists into place for Responsible AI across the AI pipeline.

In 2021, we believe we’ll see more organisations put this research and work into practice. There’s no longer a need to convince people that this is the way to go, as they’ve already gotten there. Now, it’s going to be a matter of bringing organisations the expertise to implement the ethical use of AI across their existing and future use cases.

Embracing these AI trends will not only accelerate organisations’ post-COVID recovery, but the adoption of enterprise-wide AI as well.