Search for a command to run...
I'm a DevOps Engineer with a passion for exploring emerging technologies and turning innovative ideas into practical, scalable systems with a foundation in automation, CI/CD, cloud infrastructure, and containerization.
Areas of Interest: 🔧 CI/CD Pipelines | ☁️ Cloud Platforms | 🐳 Containers & Orchestration (Docker, Kubernetes) | 🧪 Infrastructure as Code | 🔍 Monitoring & Observability | 🧠 Emerging Tech & Innovation
No comments yet. Be the first to comment.
My Journey Building with Lovable
Modern Security Modern security scanning has evolved significantly beyond the limitations of static templates. In today's complex digital landscape, effective vulnerability detection often demands the
I had the exciting opportunity to participate in New Relic’s AWS Gameday on the 18th of June, 2025. This event was a thrilling and fast-paced competition that gathered a diverse group of individuals, including tech enthusiasts, cloud engineers, and m...
INTRODUCTION Lovable provides an easy-to-use platform that lets you create a fully working website in just a few minutes. Features AI-Driven Code Generation Full-Stack Integration: Seamlessly Visual Editing Interface GitHub Synchronization One-C...
I had the wonderful opportunity to attend StrimziCon on the 4th of June, 2025. StrimziCon is a conference dedicated to the Strimzi project, which focuses on running Apache Kafka on Kubernetes giving the forum for developers, DevOps engineers, and solution architects to learn more about Strimzi and event streaming.
It started off with Keynote address, current project status, improvements, and inviting contributors who are interested to bring in the features they would love to see Strimzi have.
Each session had a 10 minutes coffee break in between and a question to reflect on, giving the attendees the time to interact and get to know their journey with Strimzi so far.
Day 2 Operations of Managing a Large Fleet of Kafka Clusters
Managing a large fleet of Kafka clusters involves a variety of complex tasks that go beyond initial deployment. These tasks include monitoring cluster health, ensuring data integrity, and maintaining optimal performance. Day 2 operations require a strategic approach to handle the ongoing management and scaling of these clusters efficiently.
Things to Watch Out For
When managing Kafka clusters, there are several critical factors to keep in mind. These include monitoring resource usage to prevent bottlenecks, ensuring data replication to avoid data loss, and keeping an eye on network latency to maintain smooth data flow. Additionally, it's important to regularly check for any security vulnerabilities and apply patches as needed to protect the clusters from potential threats.
How to Do the Upgrades Better: A Phased Process
Upgrading Kafka clusters can be a daunting task, but a phased upgrade process can help mitigate risks and ensure a smooth transition. This involves planning the upgrade in stages, starting with non-critical components to test the process. Once the initial upgrades are successful, gradually move to more critical parts of the system. Throughout the process, it's essential to have a rollback plan in place in case any issues arise, and to communicate clearly with all stakeholders to minimize disruptions.
Github: https://github.com/tomncooper/strimzi-phased-upgrade
Tina provided a comprehensive explanation on how to migrate a Strimzi-operated Kafka cluster from a ZooKeeper-based setup to a KRaft-based architecture. This process is crucial for those looking to leverage the benefits of KRaft, which simplifies the Kafka architecture by removing the dependency on ZooKeeper. Prior to the release of Strimzi 0.40.0, the transition between these two architectures was managed using the strimzi.io/kraft annotation within the Kafka custom resource. By setting this annotation to disabled, users could define a ZooKeeper-based cluster, whereas setting it to enabled would configure a KRaft-based cluster.
With the advent of Strimzi 0.40.0, the functionality of the strimzi.io/kraft annotation has been expanded to include two new values, enhancing the flexibility and control over the migration process. The migration value initiates the migration process, allowing users to transition their cluster from ZooKeeper to KRaft seamlessly. This is particularly beneficial for those seeking to modernize their Kafka infrastructure without significant downtime or disruption.
Additionally, the rollback value provides a safety net by enabling users to revert the migration if needed. This rollback capability ensures that any unforeseen issues during the migration can be addressed promptly, maintaining the stability and reliability of the Kafka environment. Overall, these enhancements in Strimzi 0.40.0 offer a more robust and user-friendly approach to managing Kafka clusters, empowering administrators to adopt KRaft with confidence.
Colt provided an in-depth exploration of the various challenges associated with testing Kafka in a local development environment. He explained that setting up Kafka locally can be complex due to its distributed nature and the need for multiple components to interact seamlessly. Colt highlighted how LittleHorse can play a crucial role in simplifying this process. By using LittleHorse, developers can create a more manageable and efficient local testing setup, which helps bridge the gap between local development and production environments. This approach not only streamlines the testing process but also ensures that developers can identify and resolve potential issues early in the development cycle, leading to more robust and reliable Kafka deployments.
Demo: https://github.com/coltmcnealy-lh/strimzicon-demo-2025
These sessions provided valuable insights into best practices, innovative solutions, and emerging trends in the world of Kafka and Kubernetes. If you are interested to check out the playlist from 2024, here you go: Strimzicon 2024
