Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[egressGateway] NATed while the destination is on ipv4NativeRoutingCIDR #38221

Open
2 of 3 tasks
pasztorl opened this issue Mar 16, 2025 · 1 comment
Open
2 of 3 tasks

[egressGateway] NATed while the destination is on ipv4NativeRoutingCIDR #38221

pasztorl opened this issue Mar 16, 2025 · 1 comment
Labels
kind/bug This is a bug in the Cilium logic. kind/community-report This was reported by a user in the Cilium community, eg via Slack. needs/triage This issue requires triaging to establish severity and next steps.

Comments

@pasztorl
Copy link

Is there an existing issue for this?

  • I have searched the existing issues

Version

equal or higher than v1.17.1 and lower than v1.18.0

What happened?

cilium bpf nat list

TCP OUT10.15.0.8:443 -> 10.15.5.1:58471 XLATE_SRC 80.77.124.102:50034 Created=465sec ago NeedsCT=1

where
10.15.0.8 is the pod
10.15.5.1 is a vpn client on wg1 interface
ipv4NativeRoutingCIDR set to 10.0.0.0/8

The packet then never goes through on wg1.
If i delete the CilliumEgressPolicy they can communicate.

I think CilliumEgressPolicy should not NAT packets targeted to ipv4NativeRoutingCIDR. Am I miss something?

How can we reproduce the issue?

helm values:

autoDirectNodeRoutes: true
bandwidthManager:
  enabled: true
bgpControlPlane:
  secretsNamespace:
    create: false
    name: network
bpf:
  datapathMode: netkit
  hostLegacyRouting: false
  lbExternalClusterIP: true
  masquerade: true
cluster:
  id: 1
  name: test-example-com
clusterDomain: k8s.test.example.com
cni:
  exclusive: false
dashboards:
  enabled: true
devices: public private wg+
egressGateway:
  enabled: true
enableIPv4Masquerade: true
enableRuntimeDeviceDetection: true
endpointRoutes:
  enabled: true
envoyConfig:
  secretsNamespace:
    create: false
    name: network
externalIPs:
  enabled: true
gatewayAPI:
  enabled: false
  secretsNamespace:
    create: false
    name: network
hostFirewall:
  enabled: true
hostPort:
  enabled: true
ingressController:
  enabled: false
  secretsNamespace:
    create: false
    name: network
ipam:
  mode: multi-pool
  operator:
    autoCreateCiliumPodIPPools:
      default:
        ipv4:
          cidrs:
          - 10.207.0.0/19
          maskSize: 28
    dashboards:
      enabled: true
      namespace: monitoring
    multiPoolPreAllocation: true
    prometheus:
      enabled: true
      labels:
        release: prometheus
      metricRelabelings: null
ipv4NativeRoutingCIDR: 10.0.0.0/8
ipv6:
  enabled: false
k8s:
  requireIPv4PodCIDR: true
  requireIPv6PodCIDR: false
k8sServiceHost: kube-api.test.example.com
k8sServicePort: 443
kubeProxyReplacement: true
l7Proxy: true
loadBalancer:
  acceleration: best-effort
  algorithm: maglev
  mode: dsr
nodePort:
  enabled: true
operator:
  dashboards:
    enabled: true
    namespace: monitoring
  prometheus:
    enabled: true
    interval: 60s
    labels:
      release: prometheus
    metricRelabelings: null
  replicas: 1
  rollOutPods: true
prometheus:
  enabled: true
  serviceMonitor:
    enabled: true
    interval: 60s
    labels:
      release: prometheus
routing-mode: native
routingMode: native
secretsNamespace:
  create: false
  sync: false
socketLB:
  enabled: true
  hostNamespaceOnly: true
tls:
  secretsNamespace:
    create: false
    name: network

CiliumPodIPPool:

apiVersion: cilium.io/v2alpha1
kind: CiliumPodIPPool
metadata:
  name: test-example-com
spec:
  ipv4:
    cidrs:
      - 10.15.0.0/22
    maskSize: 27

EgressGatewayPolicy:

apiVersion: cilium.io/v2
kind: CiliumEgressGatewayPolicy
metadata:
  name: '$NAMESPACE-$SERVICE_NAME'
spec:
  selectors:
    - podSelector:
        matchLabels:
          io.kubernetes.pod.namespace: '$NAMESPACE'
  destinationCIDRs:
    - "0.0.0.0/0"
  egressGateway:
    nodeSelector:
      matchLabels:
        kubernetes.io/hostname: '$HOSTNAME'
    egressIP: '$IP'

Cilium Version

1.17.2

Kernel Version

6.8.0-55

Kubernetes Version

1.32.2

Regression

No response

Sysdump

No response

Relevant log output



Anything else?


No response


Cilium Users Document


    

  •  Are you a user of Cilium? Please add yourself to the Users doc
    • 



Code of Conduct


    

  •  I agree to follow this project's Code of Conduct
    • 


      		
    

      
  





        

            


            

              
            

        

      


      
    








      

    



      

  
            

    

      
    

    


      

          @pasztorl
pasztorl




            added
      

  kind/bug

  This is a bug in the Cilium logic.

      

  kind/community-report

  This was reported by a user in the Cilium community, eg via Slack.

      

  needs/triage

  This issue requires triaging to establish severity and next steps.

  labels


      Mar 16, 2025

    

  








      

  
      



  

  @pasztorl





  


    

      

  

    

        
    
    

  


      
          
  
      
            Copy link

  

      
    

  


  

      



      

  Author


  


  

    

  





      


        


  
    

      
          
Update: if i set this in EgressGatewayPolicy


...
  excludedCIDRs:
    - 10.0.0.0/8



it works fine, but question remains: should EgressGatewayPolicy handle ipv4NativeRoutingCIDR?

      		
    

  





        


            

            

              
            

        

      


      

            
  

    
  
    
    

  

  



    
















  
  

    

      

  





  




  
  

              

  
    Sign up for free
    to join this conversation on GitHub.
    Already have an account?
    Sign in to comment


  



  






  
                  




    

  


      
  

    Assignees
  



        

    No one assigned







      


  


  

    Labels
  



    

      

    kind/bug

  This is a bug in the Cilium logic.
  

    kind/community-report

  This was reported by a user in the Cilium community, eg via Slack.
  

    needs/triage

  This issue requires triaging to establish severity and next steps.








      

  

    

        

    Projects
  


        




    None yet




  



      


  

    
  

    Milestone
  


      No milestone





    
      
          


  

    

      
        

          
    

    Development
  




            
    
No branches or pull requests







      
    

  




      

    

  
      

  

    

      1 participant
    

    

        
          @pasztorl