Zeanox
Zeanox6mo ago

Proxmox and cloudflared Tunnelling

Hi! Currently i am trying to configure one of my machines, proxmox is up and running and uses the ip. Id like it to use my domain so i am using cloudflared tunneling. I add the public hostname and the ip of the server yet its not working! As well as this eventually all of my externally hosted servers will use proxmox and cloudflared tunneling. How would i connect them?
No description
No description
93 Replies
Wagon
Wagon6mo ago
Show the private network tab as well Also under the public hostname entry make sure this option is selected
Wagon
Wagon6mo ago
No description
Zeanox
ZeanoxOP6mo ago
Theres nothing under private network at the moment And this is fixed! All working now however im having another issue @Wagon Im trying to connect ProxmoxM1 Which is hosted in my house to ProxmoxM2 which is hosted via hetzner. All the guides are for like if proxmox are on the same network Is there any fix to this?
Wagon
Wagon6mo ago
Are you trying to cluster them? or just access one from the other If you are trying to cluster them. Really your only option is setup a VPN server at your house with something like wire guard and connect the two. If you have full control over both networks a site to site vpn would be optimal
Wagon
Wagon6mo ago
Proxmox Support Forum
Add node to a cluster from another network
Hi, I'm kinda new to proxmox and i wanted to know if it is possible to add my proxmox server in my friend's cluster who's in another network ? And if this is indeed possible, how can i do it ? Here's some screenshots so you can see how the issue look like. Thank you very much for reading my...
Zeanox
ZeanoxOP6mo ago
Im tryna have it so i only have to use one domain and control both nodes from that etc. The vms dont need to share resources between eachother just come up on one proxmox instance and i carry out changes etc
Wagon
Wagon6mo ago
Yeah then you want to cluster them
Zeanox
ZeanoxOP6mo ago
So that means doing the whole linked vpn between them both etc
Wagon
Wagon6mo ago
Either a site to site or hosting a VPN server yeah
Zeanox
ZeanoxOP6mo ago
Got it will take a look around and see what i can do 🙂 thanks! Cant even get proxmox on hetzner working atm Set up the bridge and containers are saying there is no network
Wagon
Wagon6mo ago
linux VM? if so send the output of cat /etc/resolv.conf
Zeanox
ZeanoxOP6mo ago
So atm just trying to deploy the cloudflared tunnel think it uses debian so ill have a look now root@proxmoxm2 ~ # cat /etc/resolv.conf Hetzner Online GmbH installimage nameserver config nameserver 185.12.64.1 nameserver 2a01:4ff:ff00::add:2 nameserver 185.12.64.2 nameserver 2a01:4ff:ff00::add:1 root@proxmoxm2 ~ #
Wagon
Wagon6mo ago
also do ip route show Also I need to see cat /etc/resolv.conf in the VM/container having issues
Zeanox
ZeanoxOP6mo ago
default via 95.216.34.1 dev vmbr0 proto kernel onlink 95.216.34.0/26 dev vmbr0 proto kernel scope link src 95.216.34.25 ah okay give me a second
Zeanox
ZeanoxOP6mo ago
cant even get into the console that the container creates
No description
Zeanox
ZeanoxOP6mo ago
but from the install script for cloudflared thats the responce
Zeanox
ZeanoxOP6mo ago
okay more concerningly got an email from hetzner
No description
No description
Wagon
Wagon6mo ago
ok yeah I will need to see cat /etc/resolv.conf cat /etc/network/interfaces of the conatiner
Zeanox
ZeanoxOP6mo ago
let me make a just fresh debian instance and try that because atm for the container i cant even access the shell for it
Wagon
Wagon6mo ago
wait can you send me the entire output of ip addr on proxmox host....
Wagon
Wagon6mo ago
That is a common mistake with proxmox on hetzner servers. You must not using a bridge to your public interfaces. You have to use a routet solution. Background: pve creates another virtial device (for firewalling) with random mac adresses for each host and put in the bridge too. So hetzner will detect more than the allowed mac adresses. I found no way to prevent this, it is the solution/design from pve. I had this problem too, tried different workarouds but have to switch to the routet solution. Everything is fine now. I did'nt found a way to prevent the creating of addition devices for each VM. If you MUST use a bridge, do not use pve, use a plain qemu ontop another distribution. You can check if you affected (in your bridged) setup by check the devices in your bridge (assume this is vmbr0) https://forum.proxmox.com/threads/mac-address-abuse-report.95656/post-446006
Proxmox Support Forum
MAC Address abuse report?!
Hi, Here we go again :) We are using Proxmox 7.1-8 and got an abuse message from Hetzner today: Abuse Message [AbuseID:the_ip]: MAC-Errors: MAC-Report for #server_id(server_ip) Kontaktfoto From [email protected] on 2021-12-24 13:06 Detaljer Abuse Message 9B839A1A.txt (~194 B) Dear Mr...
Zeanox
ZeanoxOP6mo ago
yeah even for a fesh debian instance it wont create as its not using dhcp protocol 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo valid_lft forever preferred_lft forever inet6 ::1/128 scope host noprefixroute valid_lft forever preferred_lft forever 2: enp27s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master vmbr0 state UP group default qlen 1000 link/ether 30:9c:23:63:41:1c brd ff:ff:ff:ff:ff:ff 3: vmbr0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000 link/ether 30:9c:23:63:41:1c brd ff:ff:ff:ff:ff:ff inet 95.216.34.25/26 scope global vmbr0 valid_lft forever preferred_lft forever inet6 fe80::329c:23ff:fe63:411c/64 scope link valid_lft forever preferred_lft forever 8: tap100i0: <BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master fwbr100i0 state UNKNOWN group default qlen 1000 link/ether 5a:6d:be:78:a7:87 brd ff:ff:ff:ff:ff:ff 9: fwbr100i0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000 link/ether 6a:98:53:bb:b5:62 brd ff:ff:ff:ff:ff:ff 10: fwpr100p0@fwln100i0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master vmbr0 state UP group default qlen 1000 link/ether 92:ca:f3:0e:4c:ce brd ff:ff:ff:ff:ff:ff 11: fwln100i0@fwpr100p0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master fwbr100i0 state UP group default qlen 1000 link/ether 6a:98:53:bb:b5:62 brd ff:ff:ff:ff:ff:ff
Zeanox
ZeanoxOP6mo ago
That will be the issue then
No description
Wagon
Wagon6mo ago
yup
Zeanox
ZeanoxOP6mo ago
So how would i be able to fix this>
Wagon
Wagon6mo ago
Apperantly disableing mac learning on that brige may help, try tossing this in the config and see if you get another abuse report https://pve.proxmox.com/pve-docs/chapter-sysadmin.html#_disabling_mac_learning_on_a_bridge:~:text=kernel%20commandline.-,Disabling%20MAC%20Learning%20on%20a%20Bridge,-By%20default%2C%20MAC
Zeanox
ZeanoxOP6mo ago
No description
Zeanox
ZeanoxOP6mo ago
Got it so it should look like that
Wagon
Wagon6mo ago
yuh
Zeanox
ZeanoxOP6mo ago
perfect the systems rebooting. After this go onto the email check the problem is fixed then deploy another container?
Wagon
Wagon6mo ago
can you send me the docs you are using to deploy this container or your steps
Zeanox
ZeanoxOP6mo ago
Proxmox VE Helper-Scripts
Scripts for Streamlining Your Homelab with Proxmox VE
Zeanox
ZeanoxOP6mo ago
Literally chuck it in grab the key from cloudflare and it works. For my home hosted machine worked no issues but once again no network
Zeanox
ZeanoxOP6mo ago
No description
Wagon
Wagon6mo ago
and yuou cant access the containers console
Zeanox
ZeanoxOP6mo ago
Ah, gave it a little and its giving me a login which i have no clue what it is
Wagon
Wagon6mo ago
wdym
Zeanox
ZeanoxOP6mo ago
No description
Zeanox
ZeanoxOP6mo ago
yeah nd now consoles gone
Zeanox
ZeanoxOP6mo ago
No description
Wagon
Wagon6mo ago
tbh. I would just make a small VM and use docker cause i aint trouble shooting this random install script
Zeanox
ZeanoxOP6mo ago
I dont think its this script its the proxmox network settings because even making a new vm im having issues and its to do with the network
Wagon
Wagon6mo ago
ok then please on the VM send me this this
Zeanox
ZeanoxOP6mo ago
But i cant get into console of the vm or container which is the problem haha because the setups failing because of the network config
Wagon
Wagon6mo ago
Badger Bite
YouTube
Hetzner proxmox network setup and DHCP server
How to setup networking on a hetzner proxmox server for NAT and briged public IP networking (required purchasing additional IP Addresses)
Zeanox
ZeanoxOP6mo ago
Ah would have to purchase a second ip tho
Wagon
Wagon6mo ago
Then you will be stuck with NAT (icky)
Wagon
Wagon6mo ago
Networking for VMs on a Hetzner Dedicated Server - s.koch blog
I wanted to start hosting VMs on my Hetzner root server. For IPv6 this should bestraight-forward since my server has a /64 subnet, for IPv4 I need NAT.If you...
Zeanox
ZeanoxOP6mo ago
I see so that second ip may be worth it and just have a container running pfsence or something
Wagon
Wagon6mo ago
Imo, yeah. Pfsense or my favorite OPNSense you will need that if you want layer 2 type routing for your vms
Zeanox
ZeanoxOP6mo ago
But it kinda seems extreme getting another ip and setting up OPNSense if im just gunna use cloudflare tunnels ykwim?
Wagon
Wagon6mo ago
I mean not really, OPNsense aint that much work also means you can setup a wireguard server which will be handy to have as backup if cloudflare eats it
Zeanox
ZeanoxOP6mo ago
But if im not mistaken cloudflared completely bypasses the firewall
Wagon
Wagon6mo ago
Well kinda, depends on your gateway. What it does is it acts as a proxy
Zeanox
ZeanoxOP6mo ago
Exactly. If i can just mask the machines ip and have everything use its own internal network and have cloudflared proxy it then no need for second ip or any issues?
Wagon
Wagon6mo ago
I mean you still need a way to handle dhcp or use the NAT solution we discussed
Zeanox
ZeanoxOP6mo ago
I see i see Going off this So brought the second ip followed everything exactly and when installing the packages for debian for the dhcp server it had no internet so and when i got into console couldnt install the dhcp server because there was no internet so im gunna try finding another guide hopefully it will be a bit more simple now i have the second ip
Zeanox
ZeanoxOP6mo ago
https://www.youtube.com/watch?v=VZTfNXFC01Y&ab_channel=GatewayITTutorials When using this tutorial as soon as i did network configuration 1
Gateway IT Tutorials
YouTube
Install Proxmox on Hetzner baremetal server and configure networking
00:00 - Intro 02:21 - Preparation 05:46 - Installing Debian Buster and Proxmox 09:12 - Network configuration (try 1) 14:07 - GUI Preparations (including firewall settings) 17:21 - Order additional IP and use it in Proxmox 19:06 - Network configuration (try 2) 21:04 - VM creation (try 2) 22:18 - Outro apt install htop ifupdown2 nano /etc/netwo...
Zeanox
ZeanoxOP6mo ago
And rebooting it just before he goes for gui prep it just wont come back up for some reason
Zeanox
ZeanoxOP6mo ago
and trying this one: https://youtu.be/dzs5oT_7uwY?si=Afzx3XNhvlpNYJXw&t=1338 Going from 22:18 im getting no internet
Badger Bite
YouTube
NEW hetzner proxmox server network, validator setup, DHCP!!
Well, here is a repeat of some topics, so some of you may find the review helpful. This covers networking setup of the Proxmox node, ubuntu installation, DHCP server setup, port forwarding on the proxmox server to the VMs using iptables. It includes setup of an odin protocol node using statesync and cosmovisor. The next video will include th...
No description
Zeanox
ZeanoxOP6mo ago
Alright ignore that Proxmox installed Network is configured Now just the hiding the ip and cloudflared but very confused where the second ip comes into play Turns out i could cancel the second ip with the way its configured however i have a question @Wagon So it has a dhcp server running, an internal network set up with a dhcp client on it. In order for me to get cloudflared to work and proxy the proxmox domain it has to use its public ip. However its making it so if someone stumbles across the ip and port we have an issue. Is there any way to get the proxmox dashboard on an internal ip so i can use cloudflared to proxy it?
Wagon
Wagon6mo ago
Im afraid im not understanding, are you saying you want proxmox host to be on that network?
Zeanox
ZeanoxOP6mo ago
Is there a way to either stop access to the panel via the ip? Or to change the proxmox host to be on the internal one
Wagon
Wagon6mo ago
Im pretty sure you could do some fire walling with the firewall hetzner has, unless you have a static IP at home i wouldnt lock it down on the proxmox host the technically best practice is to never provide public access to host like that unless its through a VPN but its tough when you renting from a DC From my research it looks like you can use hetzners stateless firewall to secure that
Zeanox
ZeanoxOP6mo ago
If im not mistaken id disallow it from incoming requests port 8006?
No description
Wagon
Wagon6mo ago
yuh
Zeanox
ZeanoxOP6mo ago
goot it
Wagon
Wagon6mo ago
and add a LLOW FOR JUST YOUR ip
Zeanox
ZeanoxOP6mo ago
not use * ? oh i see i see Even with only allowing my ip it wont allow it to use a domain
Wagon
Wagon6mo ago
why.. oh wait yeah
Zeanox
ZeanoxOP6mo ago
So some other users will need direct access to proxmox hence me using a cloudflare tunnel application to block logins and only use company email etc But Proxmox is running on 95 ip and cloudflared and dhcp is running on 162 ip So External and internal I gotta think of a way that cloudflared and proxy the proxmox external ip without people beign able to just go directly to the ip
Wagon
Wagon6mo ago
so a way you could do it is setup a VPN on that 162 and then only allow connections to proxmoxfrom there
Zeanox
ZeanoxOP6mo ago
At the moment cloudflared is running
Zeanox
ZeanoxOP6mo ago
No description
Zeanox
ZeanoxOP6mo ago
And thats working
Wagon
Wagon6mo ago
the problem here is the hosting methodlody
Zeanox
ZeanoxOP6mo ago
However people are able to go directly to the ip like if there was a way to get the proxmox dashboard running on an internal ip so its directly port forwarded Or to block users using the ip directly and only allowing domain Unfortunately hetzner just said we have no clue
Wagon
Wagon6mo ago
I mean there probably is a way but tbh I dont know a solution off the top of my head
Zeanox
ZeanoxOP6mo ago
Even chatgpt was clueless lmaoo
Wagon
Wagon6mo ago
you have no access to layer 2/3 making any good solution hard I mean maybe @w33t may have a idea This thread might be useful to you
Zeanox
ZeanoxOP6mo ago
Lots of different people doing lots of different things however one universal thing is a firewall like pfsence or opnsence with cloudflared so Even using the hetzner firewall and blocking outbound at 8006 doesnt even work, no change whatsover domain and ip both still work strange
Wagon
Wagon6mo ago
show me the rule or the whole page
Zeanox
ZeanoxOP6mo ago
Sorry did not see your message before i slept 🙂
Zeanox
ZeanoxOP6mo ago
No description
Wagon
Wagon6mo ago
Only dest port needs set not source
Zeanox
ZeanoxOP6mo ago
Still no luck
Wagon
Wagon6mo ago
it needs to be under incoming lmao not outgoing thats mb didnt even see
Zeanox
ZeanoxOP6mo ago
ahh let me change Pretty sure its working!
Kapper
Kapper6mo ago
I was reading this out of interest at the end of the day fcking acl rules lol
Zeanox
ZeanoxOP6mo ago
LMFAO Yeah unfortunately im having a few more issues but its to do with hetzner firewall
Wagon
Wagon6mo ago
Whats up
Zeanox
ZeanoxOP6mo ago
So for hetzner cloud its as simple as just allowing no rules and my cloudflare tunnel works amazingly!. Hetzer robot its a no go for some reason setting no rules means cloudflare or anything behind it no longer works