CNME Editor Mark Forker secured an exclusive interview with Fred Crehan, Regional Director at Confluent, to learn more about data in motion and Kafka, which enterprises are leveraging to deliver better experiences for their customers, the importance of their channel ecosystem – and what really differentiates the company from its market competition.
Since its inception in 2014, Confluent, under the tutelage of its CEO and co-founder Jay Kreps has gone from strength-to-strength.
Confluent has built a foundational platform for data in motion which will empower enterprises with the ability to innovative and succeed in our new digital-first economy.
The Middle East has witnessed a tsunami of digital transformation over the last two years, but it has become evident that many are struggling to cope with the demands of transformation, hence why there is such a swell in demand for platforms such as Confluent’s Apache Kafka.
CNME secured an exclusive interview with Fred Crehan, Regional Director at Confluent, to find out more about the company’s product offering and how it plans to leverage its platform to help enterprises in the Middle East overcome the challenges they face in today’s constantly evolving IT ecosystem.
Crehan is an industry veteran with over 20 years of experience in the IT sector, and in a candid interview the charismatic Englishman expressed his belief that Confluent has built something completely different to that on offer from market rivals.
“Confluent is software infrastructure, so in the same way with urban infrastructure when you think of roads, or airports, they are never the destination in itself – instead they connect things together. You have a starting point and a destination, and that’s the same with Confluent. We are the central nervous system that connects different parts of the enterprise together, and we make data available to where it is needed in real-time, so customers can deliver a better experience, and optimise their backend infrastructure”, said Crehan.
Crehan then highlighted the importance of collaboration and said that having the right partners is a critical part of their go-to market strategy – and singled out the essential role played by system integrators.
“We connect the overall IT ecosystem, so we will never be the destination, but we need data to come from somewhere, and we need data to go to somewhere, so we have partners by definition in every sort of use-case. We need to work with partners that are going to help implement the entire use-case, and that’s where our system integrators are incredibly important to help implement and build those connections together. We provide a lot of ‘out of the box’ connections as part of our platform, but the system integrators add the brawn to really make those use-cases work. System Integrators play a huge role out here in the Middle East, because there is a skills gap everywhere in IT. In addition to this, there is a tremendous appetite for digital transformation, and they play a critical part in ensuring value is delivered. They are very important to us and will always be part of our go-to market strategy”, said Crehan.
However, Crehan pointed out that Confluent doesn’t solely rely on their channel partners and outlined that they have a direct sales motion.
“We believe that we are creating a brand-new category of data infrastructure that we call data in motion. Kafka is very well-known, but there is a lot of complexities in relation to getting Kafka to scale, and that takes a certain amount of expertise to deliver that confidence and value, which is why we play a lead role in the sales motion, but we work side-by-side with our partners to ensure that value is being delivered”, said Crehan.
Crehan reinforced how Kafka has become hugely popular with the community of open-source users, and explained how that popularity has propelled the rapid adoption of Apache Kafka globally, he also spoke about the two different options they provide for customers in terms of they use Confluent.
“Kafka has a strong following and is one of the most popular open-source projects globally – and has become synonymous with digital transformation projects. It is quite easy to get started with Kafka, but you start to encounter challenges when you think about ‘how do I make this operational, or production ready within the enterprise’? There are two ways to use Confluent, there is an out of the box self-managed version of Confluent, which means you must operate the software within your own datacenter, but the responsibility of the management of Confluent lies with you. However, we do have a wide range of excellent automation tools that make it really easy to manage that, but the responsibility is still with the customer. We also have Confluent Cloud, which is a fully managed service and that takes all the operational burden away”, said Crehan.
Crehan stated that the fully managed option was the most preferable option amongst their customers, and he illustrated this by highlighting a direct quote made by one of their clients.
“There is a case study on our website with Instacart, and to quote their Data Infrastructure Lead he said, ‘We are growing so fast, we are always under-resourced. For me to go hire a bunch of engineers to babysit Kafka, I don’t have the ability to do that.’ The reason for this is because it is really difficult to hire good people in every industry, but particularly in IT. There is such a demand for IT skills, and there is a talent war that is ongoing. Hiring people to operate infrastructure is very difficult because it is an operational type of role, and a lot of engineers and developers want to be working on applications that are at the frontend and that are making a real difference in terms of generating revenue for their companies”, said Crehan.
At the Kafka Summit 2022 in London, during his keynote speech, CEO and co-founder of Confluent Jay Kreps conceded that there was still a lot of resistance towards the technology from within the IT ecosystem.
According to Crehan, the reason for this is quite simple, and that is that Kafa is ‘new’.
“Data in motion and Kafka was created to answer a real problem, and the problem that it was trying to address was how architectures were built in the enterprise, and they are generally built at the center of the database. Databases will always remain a vital part of any kind of company architecture, but there are limitations. The limitations with databases are the fact that the data is stored, and you have to go and retrieve it, and by the time you retrieve it the data is already out of date, and by the time you have had the chance to react to it, it may be already too late. It completely flips the notion of having to go and fetch data, process it somewhere and then send it somewhere else. It changes the way applications can function with Kafka, because you no longer need to fetch data from a database as you are taking away point-to-point connections as you don’t need it anymore”, said Crehan.
Crehan added that he firmly believed that they were now getting to a point where Kafka was becoming pretty much ubiquitous.
“Confluent has a strong value proposition that makes it easy to adopt Kafka, and we are also getting to a stage now, and Jay alluded to this in his keynote that there were thousands of open jobs where Kafka was mentioned. I think what that indicates is that companies are accepting that Kafka is becoming a default technology to enable digital transformation – and it’s also becoming a skill that people want to learn, and people see a career in it, so I think when you get to that sort of tipping point it takes the perceived risk out of things that are seen as new”, said Crehan.
The Middle East region is seen by many IT and tech companies as a key global market due to the progressive transformation programs being pursued in countries such as the UAE, Saudi Arabia, and Bahrain to name a few.
Confluent has now established their Middle East HQ in Dubai, and Crehan expressed his excitement at the burgeoning opportunities that exist in the IT ecosystem across the GCC region.
“There is a lot of exciting cultural, societal, and economic change happening in the Middle East, more so than anywhere else in the world. There is a huge appetite for change and digital transformation and what the UAE has achieved in such a short space of time is truly staggering. The UAE view technology and digitalisation as a core pillar of that acceleration, as are the other countries in the Middle East. The challenges that we are helping these companies, countries or economies fulfil is that we are helping them to get there faster with reduced risk, and I think that’s the beauty of Confluent as we have done this thousands of times in the past. We do solve a build versus buy conundrum as we provide solutions that address pretty much every type of scenario that companies would have thought of. The challenges in terms of scale, security, access control no other company on the planet has that visibility and that experience, so there is a benefit to doing things by yourself, but if you want to things cheaper, faster and reduce the risk and increase your success then you’re going to want to buy Confluent to support that”, Crehan.
