Autoscaling Kubernetes pods have never been more straightforward with Horizontal Pod Autoscaler (HPA). HPA added an intuitive abstraction to effortlessly scale up or down pods dynamically based on some target metrics. However, according to the documentation, the HPA fetches these target metrics from the aggregated APIs, namely
external.metrics.k8s.io. The metrics-server add-on made it even easier to autoscale pods w.r.t CPU/Mem usage as it provided the
metrics.k8s.io API that the HPA uses. For other choices of metrics, one must leverage the
custom.metrics.k8s.io or the
external.metrics.k8s.io APIs to extend the HPA functionality.
To discuss a use-case, suppose we want…
From the numerous advantages that Kubernetes offers, I particularly find it beneficial to deploy HDFS on K8s because of the ease of scalability K8s provides and requires fewer management tasks. Data is ever-increasing, and one needs to have a stable and easy horizontal scalable architecture to accommodate terabytes of data while providing fault tolerance. By having HDFS on Kubernetes, one needs to add new nodes to an existing cluster and let Kubernetes handle the configuration for the new HDFS Datanodes (as pods)!
Below is an overview of a HDFS HA setup running on Kubernetes.
Exploring the world of MapReduce, I landed on the MapReduce documentation page of MongoDB. The first thing that is mentioned there is
So here I am thinking - “should I carry-on exploring the MapReduce method of MongoDB or dive a little deep into Aggregation Pipeline?”. For my self-satisfaction, I finally thought that it would be interesting to juxtapose MapReduce with Aggregation Pipeline and compare them. …
When it comes to data encryption, there are mainly two types — Symmetric and Asymmetric. Popular encryption methods like AES uses symmetric key encryption whereas RSA uses an asymmetric approach.
Nowadays, chat applications use end-to-end encryption to protect users’ chat messages from getting compromised by hackers or by the organization itself. End-to-end encryption (E2EE) asserts encryption and decryption to take place at the clients’ devices (end destinations). This allows the ciphered texts to transmit via an insecure and public channel, such as the internet, without the messages being compromised.
Let’s look at E2EE using both asymmetric and symmetric approaches.
LOGS. When it comes to logging, developers have to go an extra mile by inserting those log lines in their application code and also make sure their infrastructure can handle huge log ingestion to maintain scalability. The benefits? Apart from debugging, logs tell us about the flow of transactions, monitor errors, generate alerts, and do Root Cause Analysis when things go haywire.
In this read, I tried to put forward certain ways on how you can set up a reliable and scalable logging architecture for your application deployed either in VMs or in Kubernetes.
Note: We should not log just…
Systems fail. Code breaks. Responses get delayed, your Prod environment sets itself on fire and a developer resigns. This and many more of these “minor” hiccups together make a software engineer’s life complete. We, as humans, always embrace changes and so does a guy working in the IT industry. We always want to work on the next big project, add that new unique feature, and set up a more robust infrastructure in place for our applications. But what we do once the code gets into production? Monitor and Maintain. Why? Because systems work in weird ways. Computer slows down at…
Disclaimer: This is not a game review but my thoughts after playing Shadow of the Colossus. Please do not read this article if you’re planning to play the game as it contains spoilers.
“Shadow of the Colossus” quickly became an influential title after its release on Playstation 2 consoles back in 2006. So much so that the developers released its remake for Playstation 3 and 4. That’s like supporting the game for two console generations! Though it caught my attention few months back, it kept slipping behind my next “buy” list. …
Few months back we started containerising our application’s micro-services and planned to do deployment on Kubernetes rather than on traditional Virtual Machines. Deploying apps on a managed Kubernetes service like Amazon EKS or Google’s GKE is relatively easy as the cluster setup and housekeeping are taken care by them and we solely focus on our application deployment. But in our organisation, we hardly use these services unless there is a strong requirement for it. Why? Because here, we have our own private cloud on OpenStack where we can leverage the creation of virtual machines and other resources. But a managed…
Kubernetes (K8s) operators are a great way to deploy and manage your Kubernetes application.
The operator is basically a construct. In a cloud-native environment, anything that can package, deploy, and manage your application in the cloud becomes an Operator.
Being a developer, it is upto us whether we want to be the operator or let software handle it.
In Kubernetes, we can leverage operators to extend, add, and manage Kubernetes' specific functionalities and automate administrative tasks as if working with a native K8s component.
There are many useful operators open-sourced by various communities that achieve specific tasks, for example, CoreOS…
Kubernetes, docker, containerisation are some of the buzzwords in today’s DevOps world.
If you are starting out a new journey as devop or if you find interest in exploring the “Cloud Native” world, then this might be a good read for you for a start — “Kubernetes in a Nutshell”
From the Kubernetes documentation, we find the following definition
“Kubernetes is a portable, extensible open-source platform for managing containerised workloads and services, that facilitates both declarative configuration and automation.”
If you’re truly new to the cloud-native environment then this might have perplexed you. …