Simple ingress from host with microk8s?

The statement "The best I've gotten so far is using MetalLB to create a load balancer." is wrong. You must to use the ingress layer for host traffic routing.

In a bare metal environment you need to configure MetalLB to allow incoming connections from the host to k8s.

First we need a test:

curl -H "Host: nginx.ioo" http://HOST_IP

What is the result?

1. Network error
2. Error 404 or 503
3. Works!!

If Network error then you need MetalLB

microk8s.enable metallb:$(curl ipinfo.io/ip)-$(curl ipinfo.io/ip) 

Run the test again.

If Network error then you have something wrong. Check host connectivity.

If error 404 (sometimes 503) then you need a ingress rule.

# ingress-service.yaml

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: ingress-service
  annotations:
    kubernetes.io/ingress.class: nginx
    nginx.ingress.kubernetes.io/rewrite-target: /
spec:
  rules:
    - host: nginx.ioo
    - http:
        paths:
          - path: /
            backend:
              serviceName: nginx-cluster-ip-service
              servicePort: 80

Last test. It should work.

Now you can use ingress to route different domains to their respective pods inside the service.

TLDR

Update the annotation to be kubernetes.io/ingress.class: public

Why

For MicroK8s v1.21, running

microk8s enable ingress

Will create a DaemonSet called nginx-ingress-microk8s-controller in the ingress namespace.

If you inspect that, there is a flag to set the ingress class:

      - args:
        ... omitted ... 
        - --ingress-class=public
        ... omitted ... 

Therefore in order to work with most examples online, you need to either

1. Remove the --ingress-class=public argument so it defaults to nginx
2. Update annotations like kubernetes.io/ingress.class: nginx to be kubernetes.io/ingress.class: public

If I understood you correctly, there are a few ways you might be looking at.

One would be MetalLB which you already mentioned.

MetalLB provides a network load-balancer implementation for Kubernetes clusters that do not run on a supported cloud provider, effectively allowing the usage of LoadBalancer Services within any cluster.

You can read the detailed implementation A pure software solution: MetalLB

Another way would be Over a NodePort Service

This approach has a few other limitations one ought to be aware of:

• Source IP address

Services of type NodePort perform source address translation by default. This means the source IP of a HTTP request is always the IP address of the Kubernetes node that received the requestfrom the perspective of NGINX.

You can also use host network

In a setup where there is no external load balancer available but using NodePorts is not an option, one can configure ingress-nginx Pods to use the network of the host they run on instead of a dedicated network namespace. The benefit of this approach is that the NGINX Ingress controller can bind ports 80 and 443 directly to Kubernetes nodes' network interfaces, without the extra network translation imposed by NodePort Services.

You have to also remember that if you edit the configuration inside the POD, it will be gone if the Pod is restarted or it crashes.

I hope this helps you to determine which way to go with your idea.