Blog

• Announcements
• Ksctl v2.6 Customization for Cilium and Flannel CNI
• Ksctl v2.7 New Cluster Summary and Recommendation
• Ksctl v2.5 Smart Region and Instance Selection
• Ksctl v2.4 Smarter Optimization with Dynamic Region Switching
• Ksctl v2.3 Release – Smarter Cost Optimization with Dynamic Region Switching
• Ksctl v2.2.1 Release
• Ksctl v2 Release
• Our Findings
• About K8s Cluster Autoscaling

Announcements

Ksctl v2.6 Customization for Cilium and Flannel CNI

🚀 Ksctl v2.6: Advanced Customization for Cilium and Flannel CNI

We are thrilled to announce the release of Ksctl v2.6! This significant update introduces Advanced Customization for Cilium and Flannel CNI, empowering you to tailor your Kubernetes clusters’ networking to match your specific requirements and operational environment.

🌟 What’s New in Ksctl v2.6?

📌 Cilium Customization Options

Version Selection

With the new Cilium customization options, you can precisely configure your Kubernetes network environment with granular control over security policies, load balancing, and network connectivity.

Customization Modes

Choose from three flexible modes of customization:

1. Default Mode: Pre-configured setup with best practices implemented for immediate deployment

   • Optimized for general use cases
   • Zero configuration required

2. Advanced Mode: Maximum flexibility for networking experts

   • Specify custom Helm chart values.yml files

3. Guided Mode: User-friendly interface with component-level control

   • Interactive menu for enabling/disabling specific features
   • Simplified configuration of complex networking options
   • Guided approach to optimizing performance and security

For comprehensive documentation on Cilium configuration options, visit our Cilium Documentation.

📌 Flannel Customization Options

Flannel, known for its simplicity and reliability, now also offers similar customization capabilities:

Version Selection

Available Customization Modes

• Default Mode: Simple, reliable configuration for general use cases
• Advanced Mode: Custom configuration through Helm values
• Guided Mode: Step-by-step configuration with recommended options

For detailed instructions on Flannel configuration, see our Flannel Documentation.

Note: These customization features work with the ksctl cluster create command when specifying your CNI preference.

💪 Benefits of CNI Customization

• Optimized Performance: Fine-tune your network for specific workloads
• Enhanced Security: Configure network policies to match your security requirements
• Resource Efficiency: Allocate network resources according to your needs
• Operational Flexibility: Adapt to different infrastructure environments
• Future-Proof Deployments: Easily upgrade or modify as requirements change

🔄 How to Upgrade to Ksctl v2.6

To upgrade your Ksctl CLI to the latest version, simply run:

ksctl self-update

This ensures you’re using the latest version with all the new features and improvements. 🚀

With Ksctl v2.6, managing Kubernetes networking has never been more flexible and powerful! Upgrade today to take advantage of these advanced CNI customization options.

📚 Additional Resources

👉 Get started with Ksctl v2.6

Ksctl v2.7 New Cluster Summary and Recommendation

🚀 Ksctl v2.7: New Cluster Summary and Recommendation

We are thrilled to announce the release of Ksctl v2.7! This significant update introduces Cluster Summary, empowering you to gain Both Debugging Information and Security Recommendations for your Kubernetes clusters.

🌟 What’s New in Ksctl v2.7?

📌 New Logging System

This New ogging system is designed to provide most minimlistic and easy to read logs. It will help you to debug your cluster in a single command.

📌 Cluster Summary and Recommendation

The new cluster summary and recommendation feature provides you with a comprehensive overview of your cluster’s configuration and offers both debugging information and security recommendations.

In our latest release, Ksctl v2.7 introduces a powerful feature that provides a comprehensive summary of your cluster’s configuration. This feature is designed to enhance both debugging and security recommendations. With a single command, $ ksctl cluster summary, users can access critical details about their cluster.

This includes insights into pod errors, node metrics, cluster events, and latency reports.

Additionally, the feature offers recommendations for improving cluster security by checking for issues like PodSecurityPolicy, NamespaceQuota, PodResourceLimits & Requests, and NodePort configurations.

This tool is invaluable for users looking to optimize their cluster’s performance and security effortlessly.

For detailed instructions on this, Head over to Cluster Summary Documentation.

🔄 How to Upgrade to Ksctl v2.7

To upgrade your Ksctl CLI to the latest version, simply run:

ksctl self-update

This ensures you’re using the latest version with all the new features and improvements. 🚀

With Ksctl v2.7, managing Kubernetes networking has never been more flexible and powerful! Upgrade today to take advantage of these advance features.

📚 Additional Resources

👉 Get started with Ksctl v2.7

Ksctl v2.5 Smart Region and Instance Selection

🚀 Ksctl v2.5: Smart selection for Region and InstanceType

We are thrilled to announce the release of Ksctl v2.5! This update introduces Smart Card Selection, an intelligent feature that helps you select the best region and instance type for your Kubernetes clusters based on your requirements.

🌟 What’s New in Ksctl v2.5?

📌 Region Selection Sorted based on Emission Optimization

Ksctl can help you choose the most efficient region based on emission metrics, ensuring that your Kubernetes clusters are not only cost-effective but also environmentally friendly.

• 🌱 Minimizes carbon footprint by prioritizing regions with lower emissions, contributing to a greener environment.
• 🚀 Automatically ranks regions using advanced emission metrics, including:
  • Direct Carbon Intensity: (Lower is better) Measures the carbon emissions from energy production.
  • LCA Carbon Intensity: (Lower is better) Evaluates the lifecycle carbon emissions of energy sources.
  • Renewable Power Percentage: (Higher is better) Highlights regions with higher renewable energy usage.
  • Low CO₂ Power Percentage: (Higher is better) Focuses on regions with a lower share of carbon-intensive power.

This intelligent selection ensures your Kubernetes clusters are deployed in regions that align with both cost-efficiency and sustainability goals.

Figure: Visualization of region selection optimization based on emission metrics.

📌 Instance Type Selection Sorted based on Cost and Embodied Emissions Optimization

Ksctl can help you choose the most efficient instance type based on cost and embodied emissions, ensuring that your Kubernetes clusters are not only cost-effective but also environmentally friendly.

• 💰 Cost Optimization: Dynamically adapts to pricing changes across instance types, reducing infrastructure costs.
• 🌱 Sustainability Focus: Prioritizes instance types with lower embodied emissions, minimizing your carbon footprint. Automatically ranks instance types using advanced metrics, including:
  • Embodied Emissions: (Lower is better) Evaluates the environmental impact of manufacturing and deploying the instance type.

This ensures your Kubernetes clusters are optimized for both cost-efficiency and environmental sustainability.

Figure: Visualization of instance type selection optimization based on cost and embodied emissions.

This feature ensures you get maximum cost efficiency and minimum emissions while maintaining performance and reliability. This feature works with the ksctl cluster create command.

🔄 How to Upgrade to Ksctl v2.5

To upgrade your Ksctl CLI to the latest version, simply run:

ksctl self-update

This ensures you’re using the latest version with all the new features and improvements. 🚀

With Ksctl v2.5, managing Kubernetes just got more efficient and cost-effective! Upgrade today and take advantage of smart region switching to optimize your cloud expenses.

👉 Get started with Ksctl v2.5

Ksctl v2.4 Smarter Optimization with Dynamic Region Switching

🚀 Ksctl v2.4: Smarter Cost & Emission Optimization for Dynamic Region Switching

We are thrilled to announce the release of Ksctl v2.4! This update introduces Dynamic Region Switching, an intelligent cost and emission optimization feature that helps you minimize infrastructure expenses by selecting the most efficient region for your Kubernetes clusters—without changing your instance type.

🌟 What’s New in Ksctl v2.4?

📌 Dynamic Region Switching for Cost & Emission Optimization

Ksctl can now switch regions to optimize costs while maintaining the same instance type. This feature ensures that your clusters run in both the most cost-effective and least emission region without manual intervention.

• 🚀 Automatic Region Optimization: Ksctl intelligently identifies and switches to the most cost-effective and environmentally friendly regions.
• 🛠️ Flexible Region Control: Prefer to stay in your current region? You can opt to keep your cluster where it is.
• 💰 Cost Savings: Dynamically adapts to pricing changes across regions, reducing your infrastructure expenses.
• 🌱 Eco-Friendly Operations: Minimizes carbon footprint by selecting regions based on:
  • Direct Carbon Intensity: (Lower is better) Measures the carbon emissions from energy production.
  • LCA (Life Cycle Assessment) Carbon Intensity: (Lower is better) Evaluates the lifecycle carbon emissions of energy sources.
  • Renewable Power Percentage: (Higher is better) Highlights regions with higher renewable energy usage.
  • Low CO₂ Power Percentage: (Higher is better) Focuses on regions with a lower share of carbon-intensive power.

This feature ensures you get maximum cost efficiency and minimum emissions while maintaining performance and reliability.

This feature works with the ksctl cluster create command.

🔄 How to Upgrade to Ksctl v2.4

To upgrade your Ksctl CLI to the latest version, simply run:

ksctl self-update

This ensures you’re using the latest version with all the new features and improvements. 🚀

With Ksctl v2.4, managing Kubernetes just got more efficient and cost-effective! Upgrade today and take advantage of smart region switching to optimize your cloud expenses.

👉 Try Ksctl v2.4 now

Ksctl v2.3 Release – Smarter Cost Optimization with Dynamic Region Switching

🚀 Ksctl v2.3: Smarter Cost Optimization with Dynamic Region Switching

We are thrilled to announce the release of Ksctl v2.3! This update introduces Dynamic Region Switching, an intelligent cost-optimization feature that helps you minimize infrastructure expenses by selecting the most cost-effective region for your Kubernetes clusters—without changing your instance type.

🌟 What’s New in Ksctl v2.3?

📌 Dynamic Region Switching for Cost Optimization

Ksctl can now automatically switch regions to optimize costs while maintaining the same instance type. This feature ensures that your clusters run in the most cost-effective region without manual intervention.

• 🚀 Automatically identifies and switches to one of the top 5 best-priced regions
• 🛠️ Optional setting: Keep your cluster in the current region if preferred
• 💰 Reduces infrastructure costs by dynamically adapting to pricing changes across regions

This feature ensures you get maximum cost efficiency while maintaining performance and reliability.

This Feature work with the ksctl cluster create command.

🔧 Fixes and Improvements

• 🌍 Expanded AWS Support – European regions are now available after fixing the pricing API
• ⚡ Upgraded Compilation – Ksctl CLI is now built with Go v1.24 for better performance and stability

🔄 How to Upgrade to Ksctl v2.3

To upgrade your Ksctl CLI to the latest version, simply run:

ksctl self-update

This ensures you’re using the latest version with all the new features and improvements. 🚀

With Ksctl v2.3, managing Kubernetes just got more efficient and cost-effective! Upgrade today and take advantage of smart region switching to optimize your cloud expenses.

👉 Try Ksctl v2.3 now

Ksctl v2.2.1 Release

Announcing Ksctl v2.2.1: Cluster efficency at whole new level 🚀

We are excited to announce the release of Ksctl v2.2.1! This release brings a host of bugfixes and new features to transform your ksctl experience.

🆕 What’s New in Ksctl v2.2.1?

Now Instance types are categorized based on the workload

No need to know which instancetype falls under which category. Just select the category and ksctl will take care of the rest.

Here is the new categorization of instance types wrt to the cluster type you have selected:

• Self managed
  • Controlplane: ComputeIntensive
  • Etcd: MemoryIntensive
  • Loadbabalancer: GeneralPurpose
  • WorkerNode: User defined (ComputeIntensive | MemoryIntensive | GeneralPurpose | Burstable)
• Managed
  • Controlplane: Managed so no option
  • WorkerNode: User defined (ComputeIntensive | MemoryIntensive | GeneralPurpose | Burstable)

All these will help increase the performance and also reduce the cost of the cluster.

Moved to ksctl/[email protected] core

it has new security fixes and bug fixes

Ksctl v2 Release

Announcing Ksctl v2: A New Era of Effortless Kubernetes Management

We’re thrilled to announce the release of Ksctl v2! This major update is designed to simplify your Kubernetes journey—whether you’re a seasoned operator or just starting out. With v2, we’ve reimagined the way you interact with your clusters, taking away the pain of endless command-line flags and help lookups, and putting a smooth, intuitive experience at your fingertips.

🤔 Why Ksctl v2?

If you’ve ever found yourself repeatedly typing ksctl create --help just to recall the right flag, you’re not alone. Ksctl v2 was born out of our commitment to make Kubernetes cluster provisioning as frictionless as possible. Here’s what we aimed to solve with this release:

• Simplified User Experience: Now, you simply specify the resource and the associated action, and Ksctl takes care of the rest—whether it’s listing VM types, regions, Kubernetes versions, or even providing cost insights on cloud resources.
• Lowering the Entry Barrier: We know that new users often struggle with the complexities of Kubernetes.
  • Ksctl v2 is designed to be developer-friendly by offering clear defaults and guided choices Currently we help to sort out suitable cloud provider instance_type, and made sure that architecture x86 is used for maximum compatibility. Though we are working on adding support for arm architecture as well. And also in future we will be adding comparision feature for cloud providers like AWS vs Azure vs GCP, arm vs x86, AWS vs Azure (From Cost, Your Application Requirement based recommendations to Sustainability). So that you can focus on utilizing amazing Kubernetes technology, not configuring, 🫡
• Unified Tooling: Expanding beyond just a cluster provisioning tool, Ksctl now incorporates enhanced capabilities for cluster management, and add-on support—all streamlined under one roof.

🆕 What’s New in Ksctl v2?

This release brings a host of improvements and new features to transform your Kubernetes management experience:

Redesign of Ksctl core: provisioner and cluster management components

🆕 New UX & CLI Improvements

v2.0.0	v1.3.1

Before you need to remember what exactly is the vm_type, region sku, version, .. etc. Now you just need to specify the cloud provider and the rest will be taken care by ksctl. just choosing from the select options, It also shows the cost of instance_type with the block storage. A great step towards making the tool more user-friendly. 🎉

Enhanced Cloud Credential Management

• Flexible Credentials: Cloud providers will now receive credentials directly from the client. This change empowers you to choose your credential source while ensuring providers get them securely via context variables. For now all credentials will be stored in your local system. for which you need to check out $ ksctl configure ... command.

Advanced Cluster Management

• New Cluster-Management Controller: We’ve introduced a controller for managing clusters, allowing you to deploy it even on clusters not created with Ksctl. Initially focused on Ksctl-created clusters, this functionality is poised to become universally applicable.
• Self-Managed Clusters: The old ha keyword is replaced with selfmanaged to clearly indicate the type of cluster you’re operating.

Networking & Node Operations

• Cilium eBPF CNI with Hubble UI: Cilium now comes with the hubble UI enabled by default, enhancing your cluster’s observability.
• Safer Node Deletion: A node drain step is now performed before any node deletion, ensuring smoother operations.

Robust State Management & Addons

• Improved State Document: We’ve migrated versioning for components like Aks, Eks, Kind, HAProxy, and Etcd to a new field, ensuring better consistency and smoother migrations.

• New Addons System: Customize your cluster with our new addons framework. Simply specify the addons you want (e.g., switching CNI from “none” to “cilium”) in your configuration, and let Ksctl handle the rest.

  Here’s a snippet of under-the-hood configuration on how ksctl helps in creating Cilium 🐝 enabled cluster:

  {
    "cluster_name": "test-e2e-local-cilium",
    "kubernetes_version": "1.32.0",
    "cloud_provider": "local",
    "storage_type": "store-local",
    "cluster_type": "managed",
    "desired_no_of_managed_nodes": 2,
    "addons": [
      {
        "label": "kind",
        "name": "none",
        "cni": true
      },
      {
        "label": "ksctl",
        "name": "cilium",
        "cni": true
      }
    ]
  }
  

New Tools & CLI Improvements

• Two New Kubernetes Controllers: Meet ksctl/ka and ksctl/kcm! These controllers manage the application stack and cluster management addons respectively.
• Streamlined CLI Operations:
  • Self-Update Command: Keeping your CLI up-to-date is now a breeze.
  • Improved Cluster Connection: When you run $ ksctl cluster connect, your kubeconfig is automatically merged into ~/.kube/config and the current context is set accordingly.
  • Bug Fixes & Enhancements: From loading cloud credentials to scaling self-managed clusters, we’ve addressed several key issues to enhance reliability.

✨ New CLI Commands

⚠️ Important Compatibility Notice

With this release, we’re making significant changes that break backward compatibility:

• Module Transition: We are transitioning to the module name github.com/ksctl/ksctl/v2.
• Cloud Provider Update: The civo cloud provider has been completely removed.

Please ensure you update your references and workflows accordingly to take full advantage of Ksctl v2’s capabilities.

In Conclusion

Ksctl v2 is more than just an update—it’s a rethinking of the Kubernetes management experience. Our goal is to empower Developers with a tool that handles the complexity behind the scenes, letting you focus on deploying and managing clusters with ease and help you all create amazing software. Whether you’re provisioning your first cluster or managing a fleet of production environments, Ksctl v2 is here to streamline your operations.

Ready to dive in? Check out the full changelog and get started with Ksctl v2 today!

Happy clustering!

The Ksctl Team

Our Findings

About K8s Cluster Autoscaling

Pod Auto-scaler

https://cloud.google.com/kubernetes-engine/docs/concepts/horizontalpodautoscaler
https://kubernetes.io/docs/tasks/run-application/horizontal-pod-autoscale

Cluster Auto-Scaler

This blog might help The Guide To Kubernetes Cluster Autoscaler by Example

• Information from slack channel (Kubernetes #karpenter, #auto-scaler)
• How does Karpenter do it?
  • Karpenter scales using pending pod pressure.
  • There are two primary controllers for scale up and scale down
  • Provisioning_trigger controller, which will create the nodeclaims or initialize a scale request.https://github.com/kubernetes-sigs/karpenter/blob/main/pkg/controllers/provisioning/controller.go#L58 , The provisioning trigger controller watches for pending pods, and in response to those pending pods it checks if it can schedule them on the existing nodes, if not, karpenter will solve scheduling by creating addtional nodeclaims. We have another set of controllers called the node lifecycle controllers that watch for these nodeclaims, and will launch vms for a given nodeclaim.
  • Disruption Controller, which will tell karpenter to scale down the nodes. For scale down, we poll the cluster every 10 seconds via the disruption controller. https://github.com/kubernetes-sigs/karpenter/blob/main/pkg/controllers/disruption/controller.go#L126 We will iterate through all of our disruption methods and see if we can initiate some disruption action based on various factors.
• How Cluster-auto-scaler does it?
  • for scale up, the CAS looks for pending pods. for scale down, it looks for under-utilized nodes (as calculated by resource usage).
  • Note that resource usage is sum(resource_requests) / node_allocatable
  • It has nothing to do with “real” utilization
  • Basically CA’s job is to make all the pods able to schedule using as few nodes as possible, all it looks at is scheduling
  • You can use HPA/VPA to update pods based on actual resource usage and that will in turn trigger CA to add/remove nodes
• Also found a paper on CO_2 efficient karpenter
  • Andreasen, J. V. (2024). Carbon Efficient Karpenter: Optimizing Kubernetes Cluster Autoscaling for Carbon Efficiency [Computer software]. https://github.com/JacobValdemar/carbon-efficient-karpenter-thesis