Creating Customer Cluster on vSphere 6.7

Hi!

I’ve followed the guides for creating a KKP master&seed-cluster using KubeOne (used examples/terraform/vsphere - template) and KKP CE:

Everything looks fine, all pods are up and running and I’m able to access the KKP-UI, to create a project and create a cluster using the following Seed-config:

apiVersion: v1
kind: Secret
metadata:
  name: kubeconfig-cluster-example
  namespace: kubermatic
type: Opaque
data:
  kubeconfig: <created-with ./kubermatic-installer convert-kubeconfig ../kubeone/kkp-master-kubeconfig > my-kubeconfig-file; base64 -w0 my-kubeconfig-file>
---
apiVersion: kubermatic.k8s.io/v1
kind: Seed
metadata:
  name: kubermatic
  namespace: kubermatic
spec:
  # these two fields are only informational
  country: AT
  location: Vienna

  # List of datacenters where this seed cluster is allowed to create clusters in
  # In this example, user cluster will be deployed in eu-central-1 on AWS.
  datacenters:
    kd-local:
      country: "AT"
      location: "Vienna"
      spec:
        vsphere:
          allow_insecure: true
          datacenter: "demo"
          datastore: "esxi3"
          endpoint: "https://photon-machine.esxi-network"
          infra_management_user:
            password: "<pwd>"
            username: "<username>"
          templates:
            centos: ""
            flatcar: "flatcar"
            rhel: ""
            sles: ""
            ubuntu: "ubuntu"
  # reference to the kubeconfig to use when connecting to this seed cluster
  kubeconfig:
    name: kubeconfig-cluster-example
    namespace: kubermatic

I’m creating a cluster in the UI, select network to attach the nodes to, datastore etc.
After klicking “Create”, no new VMs are provisioned on the vSphere center and the cluster is stuck in pending state in the UI.

kubermatic-seed-controller-manager-fb97587cf-fxcqc is logging:

{"level":"info","time":"2021-09-08T17:26:07.833Z","caller":"initialmachinedeployment/controller.go:130","msg":"cluster not healthy","worker-name":""}
{"level":"info","time":"2021-09-08T17:26:08.028Z","logger":"kubermatic_kubernetes_controller","caller":"kubernetes/cluster_controller.go:317","msg":"failed to add finalizers","worker-name":"","error":"Operation cannot be fulfilled on clusters.kubermatic.k8s.io \"pvm2jv5grw\": the object has been modified; please apply your changes to the latest version and try again","finalizers":["kubermatic.io/cleanup-etcdbackupconfigs"]}

...

A new k8s namespace for the customer-cluster is created and also some services (apiserver-external, dns-resolver, etcd, …) get created but no deployments or pods.

Does someone know how to debug this and why there are no VMs created on my vCenter.

Any hint is very appreciated.

Best Regards

Klaus

This was a networking issue. When using a local dns-server (dnsmasq) without Load Balancer it’s import to set the externalIP of the nginx-ingress-controller - service and the nodeport-proxy - service to different IP-addresses. Thank’s to @stroebitzer !

After that I got the creation of customer cluster on vSphere working with the 2.18.0-rc.0 release.