Xray, X-UI, Reality protocol, Vision flow on Oracle Cloud

Guestbook

Oracle Cloud instance, which runs Oracle Linux:

1
2
3
4
$ ssh -i .ssh/id_rsa_ -l opc 19.214.86.113

$ sudo -i
[root@apocalypse ~]#

Install X-UI:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
[root@apocalypse ~]# bash <(curl -Ls https://raw.githubusercontent.com/FranzKafkaYu/x-ui/master/install_en.sh)

...

2025-01-21 23:05:50 (100 MB/s) - ‘/usr/bin/x-ui’ saved [25637/25637]

Install/update finished need to modify panel settings out of security
are you continue,if you type n will skip this at this time[y/n]: y
please set up your username: admin
your username will be: admin
please set up your password: password
your password will be: password
please set up the panel port: 10080
your panel port is: 10080
initializing,wait some time here...
set username and password success
account name and password set down!
set port 10080 successpanel port set down!
x-ui v0.3.4.4 install finished,it is working now...

x-ui control menu usages: 
----------------------------------------------
x-ui              - Enter     control menu
x-ui start        - Start     x-ui 
x-ui stop         - Stop      x-ui 
x-ui restart      - Restart   x-ui 
x-ui status       - Show      x-ui status
x-ui enable       - Enable    x-ui on system startup
x-ui disable      - Disable   x-ui on system startup
x-ui log          - Check     x-ui logs
x-ui update       - Update    x-ui 
x-ui install      - Install   x-ui 
x-ui uninstall    - Uninstall x-ui 
x-ui geo          - Update    geo  data
----------------------------------------------

Find the root path of X-UI:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
[root@ip-237-45-6-183 ~]# x-ui

  x-ui control menu
  0. exit
————————————————
  1. install   x-ui
  2. update    x-ui
  3. uninstall x-ui
————————————————
  4. reset username
  5. reset panel
  6. reset panel port
  7. check panel info
————————————————
  8. start x-ui
  9. stop  x-ui
  10. restart x-ui
  11. check x-ui status
  12. check x-ui logs
————————————————
  13. enable  x-ui on system startup
  14. disable x-ui on system startup
————————————————
  15. enable bbr 
  16. issuse certs
  17. x-ui cron jobs
 
x-ui status: running
enable on system startup: yes
xray status: running

please input a legal number[0-16],input 7 for checking login info:7
[INF] 当前面板信息[current panel info]:
面板版本[version]: 0.3.4.4:20230717
用户名[username]: admin
密码[userpasswd]: password
监听端口[port]: 10080
根路径[rootPath]: /xui/

Add new Inbound:

X-UI - Oracle Cloud Inbound

Open X-UI panel port and VLESS port in Oracle Cloud firewall:

1
2
3
4
5
6
7
8
[root@apocalypse ~]# firewall-cmd --zone=public --permanent --add-port=10080/tcp
success

[root@apocalypse ~]# firewall-cmd --zone=public --permanent --add-port=32854/tcp
success

[root@apocalypse ~]# firewall-cmd --reload
success

Open X-UI panel port and VLESS port in Oracle Cloud Security List Ingress Rules:

X-UI - Oracle Cloud

References

Xray, X-UI, Reality protocol, Vision flow on Amazon Linux

Guestbook

AWS EC2 instance, which runs Amazon Linux:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
$ ssh -i .ssh/aws-free-tier.pem -l ec2-user ec2-203-129-56-97.ap-southeast-2.compute.amazonaws.com
   ,     #_
   ~\_  ####_        Amazon Linux 2023
  ~~  \_#####\
  ~~     \###|
  ~~       \#/ ___   https://aws.amazon.com/linux/amazon-linux-2023
   ~~       V~' '->
    ~~~         /
      ~~._.   _/
         _/ _/
       _/m/'

$ sudo -i
[root@ip-237-45-6-183 ~]#

Install X-UI:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
[root@ip-237-45-6-183 ~]# bash <(curl -Ls https://raw.githubusercontent.com/FranzKafkaYu/x-ui/master/install_en.sh)

...

2025-01-21 23:05:50 (100 MB/s) - ‘/usr/bin/x-ui’ saved [25637/25637]

Install/update finished need to modify panel settings out of security
are you continue,if you type n will skip this at this time[y/n]: y
please set up your username: admin
your username will be: admin
please set up your password: password
your password will be: password
please set up the panel port: 10080
your panel port is: 10080
initializing,wait some time here...
set username and password success
account name and password set down!
set port 10080 successpanel port set down!
x-ui v0.3.4.4 install finished,it is working now...

x-ui control menu usages: 
----------------------------------------------
x-ui              - Enter     control menu
x-ui start        - Start     x-ui 
x-ui stop         - Stop      x-ui 
x-ui restart      - Restart   x-ui 
x-ui status       - Show      x-ui status
x-ui enable       - Enable    x-ui on system startup
x-ui disable      - Disable   x-ui on system startup
x-ui log          - Check     x-ui logs
x-ui update       - Update    x-ui 
x-ui install      - Install   x-ui 
x-ui uninstall    - Uninstall x-ui 
x-ui geo          - Update    geo  data
----------------------------------------------

Change AWS security group, and open EC2 instance port 10080 to admin user’s IP address only.

Find the root path of X-UI:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
[root@ip-237-45-6-183 ~]# x-ui

  x-ui control menu
  0. exit
————————————————
  1. install   x-ui
  2. update    x-ui
  3. uninstall x-ui
————————————————
  4. reset username
  5. reset panel
  6. reset panel port
  7. check panel info
————————————————
  8. start x-ui
  9. stop  x-ui
  10. restart x-ui
  11. check x-ui status
  12. check x-ui logs
————————————————
  13. enable  x-ui on system startup
  14. disable x-ui on system startup
————————————————
  15. enable bbr 
  16. issuse certs
  17. x-ui cron jobs
 
x-ui status: running
enable on system startup: yes
xray status: running

please input a legal number[0-16],input 7 for checking login info:7
[INF] 当前面板信息[current panel info]:
面板版本[version]: 0.3.4.4:20230717
用户名[username]: admin
密码[userpasswd]: password
监听端口[port]: 10080
根路径[rootPath]: /WMa7/

In Firefox, go to http://203.129.56.97:10080/WMa7/ and login as admin user:

X-UI - Login

X-UI - Panel

Switch to the latest version Xray e.g. v24.12.31:

X-UI - Xray

Add new Inbound:

X-UI - Inbound

Open up port 32609 to the world 0.0.0.0/0 in AWS security group.

Copy inbound QR link:

X-UI - QR

X-UI - QR Link

and paste to Xray Windows client e.g. v2rayN:

X-UI - v2rayN

References

How to install and run Tailscale client on OpenWrt

Guestbook

Install Tailscale package iptables-nft and tailscale from OpenWrt console:

OpenWrt - Tailscale iptables-nft

OpenWrt - Tailscale

Enable and connect Tailscale service in OpenWrt:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
$ ssh -l root SenseWrt
root@SenseWrt's password: 

BusyBox v1.36.1 (2024-12-03 11:41:08 UTC) built-in shell (ash)

  _______                     ________        __
 |       |.-----.-----.-----.|  |  |  |.----.|  |_
 |   -   ||  _  |  -__|     ||  |  |  ||   _||   _|
 |_______||   __|_____|__|__||________||__|  |____|
          |__| W I R E L E S S   F R E E D O M
 -----------------------------------------------------
 OpenWrt 24.10.0-rc2, r28161-ea17e958b9
 -----------------------------------------------------
 
root@SenseWrt:~# tailscale up --netfilter-mode=off --advertise-routes=192.168.88.0/24 --accept-routes
Warning: netfilter=off; configure iptables yourself.
Warning: UDP GRO forwarding is suboptimally configured on eth1, UDP forwarding throughput capability will increase with a configuration change.
See https://tailscale.com/s/ethtool-config-udp-gro

To authenticate, visit:

	https://login.tailscale.com/a/98c452901c4ba

Success.

NOTE: 192.168.88.0/24 is the ip range of local network setup in OpenWrt.

Disable key expiry for OpenWrt machine in Tailscale console, then enable all OpenWrt clients access Tailscale network:

OpenWrt - Tailscale Machines

Now add Tailscale virtual network as a new interface in OpenWrt:

OpenWrt - Tailscale Network Interface

Create firewall for Tailscale virtual network interface in OpenWrt:

OpenWrt - Tailscale Firewall

Configure firewall for Tailscale virtual network interface in OpenWrt:

OpenWrt - Tailscale Firewall General Settings

NOTE: opt network is for the downstream DHCP clients.

References

  • 韩风 Talk - Tailscale 玩法之内网穿透、异地组网、全隧道模式、纯 IP 的双栈 DERP 搭建、Headscale 协调服务器搭建，用一期搞定，看一看不亏吧？https://www.youtube.com/watch?v=mgDpJX3oNvI

How to install and run ZeroTier client on OpenWrt

Guestbook

Install ZeroTier package zerotier from OpenWrt console:

OpenWrt - ZeroTier

Enable ZeroTier service in OpenWrt:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
$ ssh -l root SenseWrt
root@SenseWrt's password: 

BusyBox v1.36.1 (2024-12-03 11:41:08 UTC) built-in shell (ash)

  _______                     ________        __
 |       |.-----.-----.-----.|  |  |  |.----.|  |_
 |   -   ||  _  |  -__|     ||  |  |  ||   _||   _|
 |_______||   __|_____|__|__||________||__|  |____|
          |__| W I R E L E S S   F R E E D O M
 -----------------------------------------------------
 OpenWrt 24.10.0-rc2, r28161-ea17e958b9
 -----------------------------------------------------

root@SenseWrt:~# cat /etc/config/zerotier 

config zerotier 'global'
	# Sets whether ZeroTier is enabled or not
	option enabled 1
	# Sets the ZeroTier listening port (default 9993; set to 0 for random)
	#option port '9993'
	# Client secret (leave blank to generate a secret on first run)
	option secret ''
	# Path of the optional file local.conf (see documentation at
	# https://docs.zerotier.com/config#local-configuration-options)
	#option local_conf_path '/etc/zerotier.conf'
	# Persistent configuration directory (to perform other configurations such
	# as controller mode or moons, etc.)
	#option config_path '/etc/zerotier'
	# Copy the contents of the persistent configuration directory to memory
	# instead of linking it, this avoids writing to flash
	#option copy_config_path '1'

# Network configuration, you can have as many configurations as networks you
# want to join (the network name is optional)
config network 'earth'
	# Identifier of the network you wish to join
	option id '8ca917257083e297'
	# Network configuration parameters (all are optional, if not indicated the
	# default values are set, see documentation at
	# https://docs.zerotier.com/config/#network-specific-configuration)
	option allow_managed '1'
	option allow_global '0'
	option allow_default '0'
	option allow_dns '0'

# Example of a second network (unnamed as it is optional)
#config network
#	option id '1234567890123456'
#	option allow_managed '1'
#	option allow_global '0'
#	option allow_default '0'
#	option allow_dns '0'

Restart ZeroTier service:

1
2
$ root@SenseWrt:~# /etc/init.d/zerotier restart
Generating secret - please wait... done.

Verify ZeroTier client status, which ztks555nye is the virtual network interface created on OpenWrt:

1
2
3
root@SenseWrt:~# zerotier-cli listnetworks
200 listnetworks <nwid> <name> <mac> <status> <type> <dev> <ZT assigned ips>
200 listnetworks 8ca917257083e297 igloo.studio 9a:09:7b:41:3d:26 OK PRIVATE ztks555nye 192.168.196.176/24

Enable and authorise the new ZeroTier client OpenWrt in https://my.zerotier.com/.

Now add ZeroTier virtual network as a new interface in OpenWrt:

OpenWrt - ZeroTier Network Interface

Create firewall for ZeroTier virtual network interface in OpenWrt:

OpenWrt - ZeroTier Firewall

Configure firewall for ZeroTier virtual network interface in OpenWrt:

OpenWrt - ZeroTier Firewall General Settings

NOTE: opt network is for the downstream DHCP clients.

Add Allow-ZeroTier firewall rule:

OpenWrt - ZeroTier Firewall Rule

Restart ZeroTier service again:

1
2
$ root@SenseWrt:~# /etc/init.d/zerotier restart
Generating secret - please wait... done.

Then all OpenWrt clients can access ZeroTier network now.

References

Connecting OpenWRT to internet

Guestbook

OpenWRT is running on Proxmox. Now setting up it to connect internet.

Add new network interface OPT on eth2 adapter:

OpenWRT - OPT Interface

OpenWRT - OPT Advanced Settings

OpenWRT - OPT Firewall Settings

OpenWRT - OPT DHCP Server General Setup

OpenWRT - OPT DHCP Server Advanced Settings

OpenWRT - OPT DHCP Server IPv6 Settings

OpenWRT - OPT DHCP Server IPv6 RA Settings

OpenWRT - OPT

Add a new Firewall Zone, from network opt to wan:

OpenWRT - Firewall Zone General Settings

OpenWRT - Firewall Zone Advanced Settings

OpenWRT - Firewall Zone Conntrack Settings

OpenWRT - Firewall Zone

Add a new Firewall Traffic Rule for network opt:

OpenWRT - Firewall Traffic Rule General Settings

OpenWRT - Firewall Traffic Rule Advanced Settings

OpenWRT - Firewall Traffic Rule Time Restrictions

OpenWRT - Firewall Traffic Rule

Setup DHCP clients’ static IP addresses:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
$ ssh -l root 192.168.2.1
root@192.168.2.1's password:

BusyBox v1.36.1 (2024-09-23 12:34:46 UTC) built-in shell (ash)

  _______                     ________        __
 |       |.-----.-----.-----.|  |  |  |.----.|  |_
 |   -   ||  _  |  -__|     ||  |  |  ||   _||   _|
 |_______||   __|_____|__|__||________||__|  |____|
          |__| W I R E L E S S   F R E E D O M
 -----------------------------------------------------
 OpenWrt 23.05.5, r24106-10cc5fcd00
 -----------------------------------------------------

root@OpenWRT:~# cat /etc/config/dhcp
config dnsmasq
	option domainneeded '1'
	option localise_queries '1'
	option rebind_protection '1'
	option rebind_localhost '1'
	option local '/local/'
	option domain 'local'
	option expandhosts '1'
	option cachesize '1000'
	option authoritative '1'
	option readethers '1'
	option leasefile '/tmp/dhcp.leases'
	option resolvfile '/tmp/resolv.conf.d/resolv.conf.auto'
	option localservice '1'
	option ednspacket_max '1232'
...
config host
	option name 'TL-SX3016F'
	option ip '192.168.2.100'
	option mac '3C:52:A1:47:23:5D'

config host
	option name 'MikroTik'
	option ip '192.168.2.110'
	option mac '78:9A:18:D0:20:ED'

config domain
	option name 'TL-SX3016F'
	option ip '192.168.2.100'

config domain
	option name 'MikroTik'
	option ip '192.168.2.110'

OpenWRT - DHCP and DNS

Running OpenWRT on Proxmox

Guestbook
  • Create OpenWRT VM in Proxmox

OpenWRT - VM General

OpenWRT - VM OS

OpenWRT - VM System

OpenWRT - VM Disks

OpenWRT - VM CPU

OpenWRT - VM Memory

OpenWRT - VM Network

This is the LAN, and Firewall is DISABLED.

OpenWRT - VM Confirm

Add WAN into VM, and Firewall is DISABLED.

Detach and delete current Hard Disk.

From Proxmox Console, download the latest OpenWRT image:

1
2
3
4
5
6
7
8
9
10
root@sense:~# wget https://downloads.openwrt.org/releases/23.05.5/targets/x86/64/openwrt-23.05.5-x86-64-generic-ext4-combined.img.gz
--2024-12-03 12:20:33--  https://downloads.openwrt.org/releases/23.05.5/targets/x86/64/openwrt-23.05.5-x86-64-generic-ext4-combined.img.gz
Resolving downloads.openwrt.org (downloads.openwrt.org)... 2a04:4e42:13::644, 151.101.82.132
Connecting to downloads.openwrt.org (downloads.openwrt.org)|2a04:4e42:13::644|:443... failed: No route to host.
Connecting to downloads.openwrt.org (downloads.openwrt.org)|151.101.82.132|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 11352149 (11M) [application/octet-stream]
Saving to: ‘openwrt-23.05.5-x86-64-generic-ext4-combined.img.gz’

openwrt-23.05.5-x86-64-generic-ext4-combined.img.gz  100%[===================================================>]  10.83M  --.-KB/s  in 0.1s

Create disk image for VM:

1
2
3
4
5
6
7
root@sense:~# gunzip openwrt-23.05.5-x86-64-generic-ext4-combined.img.gz

root@sense:~# mv openwrt-23.05.5-x86-64-generic-ext4-combined.img openwrt.raw

# increase the raw disk
root@sense:~# qemu-img resize -f raw ./openwrt.raw 1024M
Image resized.

Convert OpenWRT image to VM disk:

1
2
3
4
5
6
7
8
9
# import the raw disk to OpenWRT VM
root@sense:~# qm importdisk 104 openwrt.raw local-lvm
Use of uninitialized value $dev in hash element at /usr/share/perl5/PVE/QemuServer/Drive.pm line 555.
importing disk 'openwrt.raw' to VM 104 ...
  Logical volume "vm-104-disk-0" created.
transferred 0.0 B of 1.0 GiB (0.00%)
...
transferred 1.0 GiB of 1.0 GiB (100.00%)
unused0: successfully imported disk 'local-lvm:vm-104-disk-0'

Double click the Unused Disk, then click the Add button:

OpenWRT - VM Unused Disk

OpenWRT - VM Hardware

OpenWRT - VM Boot Order

  • Configure OpenWRT

Start up VM; change the user root password; set LAN ip address temporarily to 192.168.2.3 (Default: 192.168.1.1):

OpenWRT - Console

Login OpenWRT at http://192.168.2.3/, via LAN port, and land on dashboard:

OpenWRT - Login

OpenWRT - Dashboard

  • Themes

SSH login:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
$ ssh -l root 192.168.2.3
The authenticity of host '192.168.2.3 (192.168.2.3)' can't be established.
ED25519 key fingerprint is SHA256:AggWAL1oU8+r1f84KoqpvcsYUylZOTfN0sXwHSby3b0.
This key is not known by any other names.
Are you sure you want to continue connecting (yes/no/[fingerprint])? yes
Warning: Permanently added '192.168.2.3' (ED25519) to the list of known hosts.
root@192.168.2.3's password: 
  _______                     ________        __
 |       |.-----.-----.-----.|  |  |  |.----.|  |_
 |   -   ||  _  |  -__|     ||  |  |  ||   _||   _|
 |_______||   __|_____|__|__||________||__|  |____|
          |__| W I R E L E S S   F R E E D O M
 -----------------------------------------------------
 OpenWrt 23.05.5, r24106-10cc5fcd00
 -----------------------------------------------------

BusyBox v1.36.1 (2024-09-23 12:34:46 UTC) built-in shell (ash)

OpenWRT packages configuration:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
root@OpenWrt:~# ls -al /etc/opkg
drwxr-xr-x    3 root     root          4096 Dec  6 02:46 .
drwxr-xr-x   23 root     root          4096 Dec  6 02:41 ..
-rw-r--r--    1 root     root           103 Sep 23 12:34 customfeeds.conf
-rw-r--r--    1 root     root           555 Dec  6 02:46 distfeeds.conf
drwxr-xr-x    2 root     root          4096 Sep 23 12:34 keys

root@OpenWrt:~# ls -al /var/opkg-lists/
drwxr-xr-x    2 root     root           320 Dec  6 03:15 .
drwxrwxrwt   17 root     root           440 Dec  6 03:15 ..
-rw-r--r--    1 root     root         72705 Dec  6 03:15 openwrt_base
-rw-r--r--    1 root     root           142 Dec  6 03:15 openwrt_base.sig
-rw-r--r--    1 root     root         10039 Dec  6 03:15 openwrt_core
-rw-r--r--    1 root     root           142 Dec  6 03:15 openwrt_core.sig
-rw-r--r--    1 root     root        100416 Dec  6 03:15 openwrt_kmods
-rw-r--r--    1 root     root           142 Dec  6 03:15 openwrt_kmods.sig
-rw-r--r--    1 root     root        197242 Dec  6 03:15 openwrt_luci
-rw-r--r--    1 root     root           142 Dec  6 03:15 openwrt_luci.sig
-rw-r--r--    1 root     root        516243 Dec  6 03:15 openwrt_packages
-rw-r--r--    1 root     root           142 Dec  6 03:15 openwrt_packages.sig
-rw-r--r--    1 root     root          9918 Dec  6 03:15 openwrt_routing
-rw-r--r--    1 root     root           142 Dec  6 03:15 openwrt_routing.sig
-rw-r--r--    1 root     root         58240 Dec  6 03:15 openwrt_telephony
-rw-r--r--    1 root     root           142 Dec  6 03:15 openwrt_telephony.sig

There is some issue with IPv6 support in OpenWRT when download update. Errors thrown when wan connected to a IPv6 router: 

1
2
3
4
5
6
7
8
9
10
11
12
13
root@OpenWrt:~# opkg update
Downloading https://downloads.openwrt.org/releases/23.05.5/targets/x86/64/packages/Packages.gz
*** Failed to download the package list from https://downloads.openwrt.org/releases/23.05.5/targets/x86/64/packages/Packages.gz
...
Downloading https://downloads.openwrt.org/releases/23.05.5/packages/x86_64/telephony/Packages.gz
*** Failed to download the package list from https://downloads.openwrt.org/releases/23.05.5/packages/x86_64/telephony/Packages.gz

Collected errors:
 * opkg_download: Failed to download https://downloads.openwrt.org/releases/23.05.5/targets/x86/64/packages/Packages.gz, wget returned 4.
 * opkg_download: Check your network settings and connectivity.
...
 * opkg_download: Failed to download https://downloads.openwrt.org/releases/23.05.5/packages/x86_64/telephony/Packages.gz, wget returned 4.
 * opkg_download: Check your network settings and connectivity.

Turn off IPv6 on router to workaround.

1
2
3
4
5
6
7
8
9
10
root@OpenWRT:~# opkg update
Downloading https://downloads.openwrt.org/releases/23.05.5/targets/x86/64/packages/Packages.gz
Updated list of available packages in /var/opkg-lists/openwrt_core
Downloading https://downloads.openwrt.org/releases/23.05.5/targets/x86/64/packages/Packages.sig
Signature check passed.
...
Downloading https://downloads.openwrt.org/releases/23.05.5/packages/x86_64/telephony/Packages.gz
Updated list of available packages in /var/opkg-lists/openwrt_telephony
Downloading https://downloads.openwrt.org/releases/23.05.5/packages/x86_64/telephony/Packages.sig
Signature check passed.

Install OpenWRT2020 Theme https://openwrt.org/docs/guide-user/luci/luci.themes:

1
2
3
4
root@OpenWRT:~# opkg install luci-theme-openwrt-2020
Installing luci-theme-openwrt-2020 (git-24.332.79522-a493155) to root...
Downloading https://downloads.openwrt.org/releases/23.05.5/packages/x86_64/luci/luci-theme-openwrt-2020_git-24.332.79522-a493155_all.ipk
Configuring luci-theme-openwrt-2020.

Install Argon Theme https://github.com/jerrykuku/luci-theme-argon:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
root@OpenWRT:~# opkg install luci-compat
Installing luci-compat (git-24.332.79522-a493155) to root...
Downloading https://downloads.openwrt.org/releases/23.05.5/packages/x86_64/luci/luci-compat_git-24.332.79522-a493155_all.ipk
Installing liblua5.1.5 (5.1.5-11) to root...
Downloading https://downloads.openwrt.org/releases/23.05.5/packages/x86_64/base/liblua5.1.5_5.1.5-11_x86_64.ipk
Installing lua (5.1.5-11) to root...
Downloading https://downloads.openwrt.org/releases/23.05.5/packages/x86_64/base/lua_5.1.5-11_x86_64.ipk
Installing luci-lib-nixio (git-24.332.79522-a493155) to root...
Downloading https://downloads.openwrt.org/releases/23.05.5/packages/x86_64/luci/luci-lib-nixio_git-24.332.79522-a493155_x86_64.ipk
Installing luci-lib-ip (git-24.332.79522-a493155) to root...
Downloading https://downloads.openwrt.org/releases/23.05.5/packages/x86_64/luci/luci-lib-ip_git-24.332.79522-a493155_x86_64.ipk
Installing luci-lib-jsonc (git-24.332.79522-a493155) to root...
Downloading https://downloads.openwrt.org/releases/23.05.5/packages/x86_64/luci/luci-lib-jsonc_git-24.332.79522-a493155_x86_64.ipk
Installing liblucihttp-lua (2023-03-15-9b5b683f-1) to root...
Downloading https://downloads.openwrt.org/releases/23.05.5/packages/x86_64/luci/liblucihttp-lua_2023-03-15-9b5b683f-1_x86_64.ipk
Installing luci-lib-base (git-24.332.79522-a493155) to root...
Downloading https://downloads.openwrt.org/releases/23.05.5/packages/x86_64/luci/luci-lib-base_git-24.332.79522-a493155_all.ipk
Installing libubus-lua (2023-06-05-f787c97b-1) to root...
Downloading https://downloads.openwrt.org/releases/23.05.5/packages/x86_64/base/libubus-lua_2023-06-05-f787c97b-1_x86_64.ipk
Installing ucode-mod-lua (1) to root...
Downloading https://downloads.openwrt.org/releases/23.05.5/packages/x86_64/luci/ucode-mod-lua_1_x86_64.ipk
Installing luci-lua-runtime (git-24.332.79522-a493155) to root...
Downloading https://downloads.openwrt.org/releases/23.05.5/packages/x86_64/luci/luci-lua-runtime_git-24.332.79522-a493155_x86_64.ipk
Configuring liblua5.1.5.
Configuring lua.
Configuring luci-lib-nixio.
Configuring luci-lib-ip.
Configuring luci-lib-jsonc.
Configuring liblucihttp-lua.
Configuring luci-lib-base.
Configuring libubus-lua.
Configuring ucode-mod-lua.
Configuring luci-lua-runtime.
Configuring luci-compat.

root@OpenWRT:~# opkg install luci-lib-ipkg
Installing luci-lib-ipkg (git-24.332.79522-a493155) to root...
Downloading https://downloads.openwrt.org/releases/23.05.5/packages/x86_64/luci/luci-lib-ipkg_git-24.332.79522-a493155_all.ipk
Configuring luci-lib-ipkg.

root@OpenWRT:~# wget --no-check-certificate -O luci-theme-argon.ipk https://github.com/jerrykuku/luci-theme-argon/releases/download/v2.3.1/luci-theme-argon_2.3.1_all.ipk
root@OpenWRT:~# wget --no-check-certificate -O luci-app-argon-config.ipk https://github.com/jerrykuku/luci-app-argon-config/releases/download/v0.9/luci-app-argon-config_0.9_all.ipk

There are ERRORS:

  • //usr/lib/opkg/info/luci-theme-argon.postinst: .: line 2: can’t open ‘/etc/uci-defaults/30_luci-theme-argon’: No such file or directory
  • //usr/lib/opkg/info/luci-app-argon-config.postinst: .: line 2: can’t open ‘/etc/uci-defaults/luci-argon-config’: No such file or directory

thrown when try to install these packages.

A workaround solution is modify /lib/functions.sh file, replace line: 282:

1
( [ -f "$i" ] && cd "$(dirname $i)" && . "$i" ) && rm -f "$i"

with: 

1
( [ -f "$i" ] && cd "$(dirname $i)" && . "$i" ) && echo

temporarily. Then run installation:

1
2
3
4
5
6
7
root@OpenWRT:~# opkg install luci-theme-argon.ipk 
Installing luci-theme-argon (2.3.1) to root...
Configuring luci-theme-argon.

root@OpenWRT:~# opkg install luci-app-argon-config.ipk 
Installing luci-app-argon-config (0.9) to root...
Configuring luci-app-argon-config.
  • Upgrade

To upgrade all of the OpenWRT packages:

1
root@OpenWRT:~# opkg list-upgradable | cut -f 1 -d ' ' | xargs -r opkg upgrade

References

Fixing Multimedia Audio Controller driver missing issue in Windows 11

Guestbook

Error thrown while missing Multimedia Audio Driver driver in Device Manager in Windows 11:

Windows - Multimedia Audio Controller

when runs on an Intel i3 N305 Mini PC https://www.aliexpress.com/item/1005007278560105.html

Intel Network - Mini PC

Need this handy tool to find all the missing parts:

Windows - Driver Identifier

Driver Identifier, a digital assistant for your system’s hardware, can scan the device, identifies outdated or missing drivers, and provides a customized list of updates for your specific hardware:

Windows - Drivers

References

Setting up DNS Server in OPNsense

Guestbook

Want to access Homelab network switches’ web based administrator interface in a user friendly way, i.e., access via http://TL-SX3016F.local/, http://TL-SX3008F.local/ and http://MikroTik.local/.

In TP-Link Deco, which is the main Homelab network backbone, also provides DHCP service, firstly is to reserve IP addresses for these switches:

DNS - TP-Link Deco Reservation

Then setup entries in OPNsense Unbound DNS server:

DNS - Host Overrides

Then lookup a host on local network:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
$ cat /etc/resolv.conf
#
# macOS Notice
#
# This file is not consulted for DNS hostname resolution, address
# resolution, or the DNS query routing mechanism used by most
# processes on this system.
#
# To view the DNS configuration used by this system, use:
#   scutil --dns
#
# SEE ALSO
#   dns-sd(1), scutil(8)
#
# This file is automatically generated.
#
search local
nameserver 2403:5802:8c44:3:be24:11ff:fe21:3ce0
nameserver 192.168.68.1

$ nslookup MikroTik.local
Server:		2403:5802:8c44:3:be24:11ff:fe21:3ce0
Address:	2403:5802:8c44:3:be24:11ff:fe21:3ce0#53

Name:	MikroTik.local
Address: 192.168.68.54

Enable SR-IOV in Intel 82599ES 10Gbps Ethernet Controller

Guestbook

Intel® 82599ES 10Gbps Ethernet Controller https://ark.intel.com/content/www/us/en/ark/products/41282/intel-82599es-10-gigabit-ethernet-controller.html supports SR-IOV, which is Intel® Virtualization Technology for Connectivity (VT-c) solution.

NOTE: Intel VT Virtualization Features

  • VT-x, Intel Virtualization Technology for IA-32 and Intel 64 Processors
  • VT-d, Intel Virtualization Technology for Directed I/O
  • VT-c, Intel Virtualization Technology for Connectivity

The following step by step instructions is made in Proxmox.

Intel i3 N305 Mini PC https://www.aliexpress.com/item/1005007278560105.html

Intel Network - Mini PC

equips with two 10G SFP+ (Intel 82599ES) and two 2.5G (Intel i226-V) network cards:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
root@sense:~# lspci -v 
...
01:00.0 Ethernet controller: Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
        Flags: bus master, fast devsel, latency 0, IRQ 16, IOMMU group 12
        Memory at 80a20000 (64-bit, non-prefetchable) [size=128K]
        I/O ports at 3020 [disabled] [size=32]
        Memory at 80a44000 (64-bit, non-prefetchable) [size=16K]
        Capabilities: [40] Power Management version 3
        Capabilities: [50] MSI: Enable- Count=1/1 Maskable+ 64bit+
        Capabilities: [70] MSI-X: Enable+ Count=64 Masked-
        Capabilities: [a0] Express Endpoint, MSI 00
        Capabilities: [e0] Vital Product Data
        Capabilities: [100] Advanced Error Reporting
        Capabilities: [140] Device Serial Number a8-b8-e0-ff-ff-05-96-4d
        Capabilities: [150] Alternative Routing-ID Interpretation (ARI)
        Capabilities: [160] Single Root I/O Virtualization (SR-IOV)
        Kernel driver in use: ixgbe
        Kernel modules: ixgbe

01:00.1 Ethernet controller: Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
        Flags: bus master, fast devsel, latency 0, IRQ 17, IOMMU group 13
        Memory at 80a00000 (64-bit, non-prefetchable) [size=128K]
        I/O ports at 3000 [disabled] [size=32]
        Memory at 80a40000 (64-bit, non-prefetchable) [size=16K]
        Capabilities: [40] Power Management version 3
        Capabilities: [50] MSI: Enable- Count=1/1 Maskable+ 64bit+
        Capabilities: [70] MSI-X: Enable+ Count=64 Masked-
        Capabilities: [a0] Express Endpoint, MSI 00
        Capabilities: [e0] Vital Product Data
        Capabilities: [100] Advanced Error Reporting
        Capabilities: [140] Device Serial Number a8-b8-e0-ff-ff-05-96-4d
        Capabilities: [150] Alternative Routing-ID Interpretation (ARI)
        Capabilities: [160] Single Root I/O Virtualization (SR-IOV)
        Kernel driver in use: ixgbe
        Kernel modules: ixgbe

01:10.0 Ethernet controller: Intel Corporation 82599 Ethernet Controller Virtual Function (rev 01)
        Flags: bus master, fast devsel, latency 0, IOMMU group 18
        Memory at 4017000000 (64-bit, prefetchable) [virtual] [size=16K]
        Memory at 4017100000 (64-bit, prefetchable) [virtual] [size=16K]
        Capabilities: [70] MSI-X: Enable+ Count=3 Masked-
        Capabilities: [a0] Express Endpoint, MSI 00
        Capabilities: [100] Advanced Error Reporting
        Capabilities: [150] Alternative Routing-ID Interpretation (ARI)
        Kernel driver in use: vfio-pci
        Kernel modules: ixgbevf

...

02:00.0 Ethernet controller: Intel Corporation Ethernet Controller I226-V (rev 04)
        Subsystem: Intel Corporation Ethernet Controller I226-V
        Flags: bus master, fast devsel, latency 0, IRQ 18, IOMMU group 14
        Memory at 80600000 (32-bit, non-prefetchable) [size=1M]
        Memory at 80700000 (32-bit, non-prefetchable) [size=16K]
        Capabilities: [40] Power Management version 3
        Capabilities: [50] MSI: Enable- Count=1/1 Maskable+ 64bit+
        Capabilities: [70] MSI-X: Enable+ Count=5 Masked-
        Capabilities: [a0] Express Endpoint, MSI 00
        Capabilities: [100] Advanced Error Reporting
        Capabilities: [140] Device Serial Number a8-b8-e0-ff-ff-05-96-4f
        Capabilities: [1c0] Latency Tolerance Reporting
        Capabilities: [1f0] Precision Time Measurement
        Capabilities: [1e0] L1 PM Substates
        Kernel driver in use: igc
        Kernel modules: igc

03:00.0 Ethernet controller: Intel Corporation Ethernet Controller I226-V (rev 04)
        Subsystem: Intel Corporation Ethernet Controller I226-V
        Flags: bus master, fast devsel, latency 0, IRQ 16, IOMMU group 15
        Memory at 80400000 (32-bit, non-prefetchable) [size=1M]
        Memory at 80500000 (32-bit, non-prefetchable) [size=16K]
        Capabilities: [40] Power Management version 3
        Capabilities: [50] MSI: Enable- Count=1/1 Maskable+ 64bit+
        Capabilities: [70] MSI-X: Enable+ Count=5 Masked-
        Capabilities: [a0] Express Endpoint, MSI 00
        Capabilities: [100] Advanced Error Reporting
        Capabilities: [140] Device Serial Number a8-b8-e0-ff-ff-05-96-50
        Capabilities: [1c0] Latency Tolerance Reporting
        Capabilities: [1f0] Precision Time Measurement
        Capabilities: [1e0] L1 PM Substates
        Kernel driver in use: igc
        Kernel modules: igc

04:00.0 Network controller: Intel Corporation Wi-Fi 7(802.11be) AX1775*/AX1790*/BE20*/BE401/BE1750* 2x2 (rev 1a)
        Subsystem: Intel Corporation Wi-Fi 7(802.11be) AX1775*/AX1790*/BE20*/BE401/BE1750* 2x2 (BE200 320MHz [Gale Peak])
        Flags: bus master, fast devsel, latency 0, IRQ 17, IOMMU group 16
        Memory at 80900000 (64-bit, non-prefetchable) [size=16K]
        Capabilities: [40] Power Management version 3
        Capabilities: [50] MSI: Enable- Count=1/1 Maskable- 64bit+
        Capabilities: [70] Express Endpoint, MSI 00
        Capabilities: [b0] MSI-X: Enable+ Count=32 Masked-
        Capabilities: [100] Advanced Error Reporting
        Capabilities: [148] Secondary PCI Express
        Capabilities: [158] Physical Layer 16.0 GT/s <?>
        Capabilities: [17c] Lane Margining at the Receiver <?>
        Capabilities: [188] Latency Tolerance Reporting
        Capabilities: [190] L1 PM Substates
        Capabilities: [1a0] Vendor Specific Information: ID=0002 Rev=4 Len=100 <?>
        Capabilities: [2a0] Data Link Feature <?>
        Capabilities: [2ac] Precision Time Measurement
        Capabilities: [2b8] Vendor Specific Information: ID=0003 Rev=1 Len=054 <?>
        Capabilities: [500] Vendor Specific Information: ID=0023 Rev=1 Len=010 <?>
        Kernel driver in use: iwlwifi
        Kernel modules: iwlwifi
...

root@sense:~# ip a
...
2: enp2s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master vmbr2 state UP group default qlen 1000
    link/ether a8:b8:e0:05:96:4f brd ff:ff:ff:ff:ff:ff
3: enp3s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master vmbr3 state UP group default qlen 1000
    link/ether a8:b8:e0:05:96:50 brd ff:ff:ff:ff:ff:ff
4: enp1s0f0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master vmbr0 state UP group default qlen 1000
    link/ether a8:b8:e0:05:96:4d brd ff:ff:ff:ff:ff:ff
5: enp1s0f1: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq master vmbr1 state DOWN group default qlen 1000
    link/ether a8:b8:e0:05:96:4e brd ff:ff:ff:ff:ff:ff
7: enp1s0f0v1: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
    link/ether 7a:de:19:c7:c2:ce brd ff:ff:ff:ff:ff:ff    
...
22: wlp4s0f0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
    link/ether e0:8f:4c:b2:58:95 brd ff:ff:ff:ff:ff:ff

Verify Intel 82599ES network card status:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
root@sense:~# ethtool enp1s0f0
Settings for enp1s0f0:
        Supported ports: [ FIBRE ]
        Supported link modes:   10000baseT/Full
        Supported pause frame use: Symmetric
        Supports auto-negotiation: No
        Supported FEC modes: Not reported
        Advertised link modes:  10000baseT/Full
        Advertised pause frame use: Symmetric
        Advertised auto-negotiation: No
        Advertised FEC modes: Not reported
        Speed: 10000Mb/s
        Duplex: Full
        Auto-negotiation: off
        Port: FIBRE
        PHYAD: 0
        Transceiver: internal
        Supports Wake-on: d
        Wake-on: d
        Current message level: 0x00000007 (7)
                               drv probe link
        Link detected: yes

root@sense:~# ethtool -i enp1s0f0
driver: ixgbe
version: 6.8.12-2-pve
firmware-version: 0x800003de
expansion-rom-version: 
bus-info: 0000:01:00.0
supports-statistics: yes
supports-test: yes
supports-eeprom-access: yes
supports-register-dump: yes
supports-priv-flags: yes

Enable IOMMU in Proxmox:

1
2
root@sense:~# cat /etc/default/grub | grep GRUB_CMDLINE_LINUX_DEFAULT
GRUB_CMDLINE_LINUX_DEFAULT="quiet intel_iommu=on iommu=pt"

IOMMU PT mode improves the performance of other PCIe devices in the system when passthrough is being used.

Update GRUB:

1
2
3
4
5
6
7
8
9
root@sense:~# update-grub
Generating grub configuration file ...
Found linux image: /boot/vmlinuz-6.8.12-2-pve
Found initrd image: /boot/initrd.img-6.8.12-2-pve
Found linux image: /boot/vmlinuz-6.8.4-2-pve
Found initrd image: /boot/initrd.img-6.8.4-2-pve
Found memtest86+ 64bit EFI image: /boot/memtest86+x64.efi
Adding boot menu entry for UEFI Firmware Settings ...
done

Reboot Proxmox and verify IOMMU is enabled:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
root@sense:~# dmesg | grep -i IOMMU
[    0.000000] Command line: BOOT_IMAGE=/boot/vmlinuz-6.8.12-2-pve root=/dev/mapper/pve-root ro quiet intel_iommu=on iommu=pt
[    0.053988] Kernel command line: BOOT_IMAGE=/boot/vmlinuz-6.8.12-2-pve root=/dev/mapper/pve-root ro quiet intel_iommu=on iommu=pt
[    0.054043] DMAR: IOMMU enabled
[    0.145258] DMAR-IR: IOAPIC id 2 under DRHD base  0xfed91000 IOMMU 1
[    0.403235] pci 0000:00:02.0: DMAR: Skip IOMMU disabling for graphics
[    0.440538] iommu: Default domain type: Translated
[    0.440538] iommu: DMA domain TLB invalidation policy: lazy mode
[    0.483908] DMAR: IOMMU feature fl1gp_support inconsistent
[    0.483909] DMAR: IOMMU feature pgsel_inv inconsistent
[    0.483910] DMAR: IOMMU feature nwfs inconsistent
[    0.483911] DMAR: IOMMU feature dit inconsistent
[    0.483912] DMAR: IOMMU feature sc_support inconsistent
[    0.483912] DMAR: IOMMU feature dev_iotlb_support inconsistent
[    0.493686] pci 0000:00:02.0: Adding to iommu group 0
[    0.493726] pci 0000:00:00.0: Adding to iommu group 1
[    0.493739] pci 0000:00:0d.0: Adding to iommu group 2
[    0.493764] pci 0000:00:14.0: Adding to iommu group 3
[    0.493774] pci 0000:00:14.2: Adding to iommu group 3
[    0.493786] pci 0000:00:16.0: Adding to iommu group 4
[    0.493793] pci 0000:00:17.0: Adding to iommu group 5
[    0.493806] pci 0000:00:1c.0: Adding to iommu group 6
[    0.493820] pci 0000:00:1c.6: Adding to iommu group 7
[    0.493831] pci 0000:00:1d.0: Adding to iommu group 8
[    0.493842] pci 0000:00:1d.1: Adding to iommu group 9
[    0.493859] pci 0000:00:1d.3: Adding to iommu group 10
[    0.493878] pci 0000:00:1f.0: Adding to iommu group 11
[    0.493886] pci 0000:00:1f.3: Adding to iommu group 11
[    0.493895] pci 0000:00:1f.4: Adding to iommu group 11
[    0.493903] pci 0000:00:1f.5: Adding to iommu group 11
[    0.493918] pci 0000:01:00.0: Adding to iommu group 12
[    0.493934] pci 0000:01:00.1: Adding to iommu group 13
[    0.493945] pci 0000:02:00.0: Adding to iommu group 14
[    0.493956] pci 0000:03:00.0: Adding to iommu group 15
[    0.493982] pci 0000:04:00.0: Adding to iommu group 16
[    0.493993] pci 0000:05:00.0: Adding to iommu group 17
[    3.684070] pci 0000:01:10.1: Adding to iommu group 18
[    3.684303] pci 0000:01:10.3: Adding to iommu group 19
[    3.684501] pci 0000:01:10.5: Adding to iommu group 20
[    3.684685] pci 0000:01:10.7: Adding to iommu group 21
[    3.684885] pci 0000:01:11.1: Adding to iommu group 22
[    3.685096] pci 0000:01:11.3: Adding to iommu group 23
[    3.685296] pci 0000:01:11.5: Adding to iommu group 24
[    3.685473] pci 0000:01:11.7: Adding to iommu group 25
[    3.755233] pci 0000:01:10.0: Adding to iommu group 26
[    3.755462] pci 0000:01:10.2: Adding to iommu group 27
[    3.755761] pci 0000:01:10.4: Adding to iommu group 28
[    3.755939] pci 0000:01:10.6: Adding to iommu group 29
[    3.756107] pci 0000:01:11.0: Adding to iommu group 30
[    3.756276] pci 0000:01:11.2: Adding to iommu group 31
[    3.756444] pci 0000:01:11.4: Adding to iommu group 32
[    3.756628] pci 0000:01:11.6: Adding to iommu group 33

ATTENTION: IOMMU Group not order by PCI BUS IDs

This is an ISSUE CAN’T setup PCI Device Resource Mappings, because of errors, e.g., “Configuration for iommugroup not correct (‘21’ != ‘’27)” thrown, sometime, after Proxmox reboot!

Work around solution is go to Resource Mappings in Proxmox, edit, make no change and save:

Intel Network - Resource Mappings

Filter out network card udev information, e.g., enp1s0f1:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
root@sense:~# udevadm info -e 
...
P: /devices/pci0000:00/0000:00:1c.0/0000:01:00.1/net/enp1s0f1
M: enp1s0f1
R: 1
U: net
I: 5
E: DEVPATH=/devices/pci0000:00/0000:00:1c.0/0000:01:00.1/net/enp1s0f1
E: SUBSYSTEM=net
E: INTERFACE=enp1s0f1
E: IFINDEX=5
E: USEC_INITIALIZED=2364228
E: ID_NET_NAMING_SCHEME=v252
E: ID_NET_NAME_MAC=enxa8b8e005964e
E: ID_OUI_FROM_DATABASE=Changwang Technology inc.
E: ID_NET_NAME_PATH=enp1s0f1
E: ID_BUS=pci
E: ID_VENDOR_ID=0x8086
E: ID_MODEL_ID=0x10fb
E: ID_PCI_CLASS_FROM_DATABASE=Network controller
E: ID_PCI_SUBCLASS_FROM_DATABASE=Ethernet controller
E: ID_VENDOR_FROM_DATABASE=Intel Corporation
E: ID_MODEL_FROM_DATABASE=82599ES 10-Gigabit SFI/SFP+ Network Connection
E: ID_PATH=pci-0000:01:00.1
E: ID_PATH_TAG=pci-0000_01_00_1
E: ID_NET_DRIVER=ixgbe
E: ID_NET_LINK_FILE=/usr/lib/systemd/network/99-default.link
E: ID_NET_NAME=enp1s0f1
E: SYSTEMD_ALIAS=/sys/subsystem/net/devices/enp1s0f1
E: TAGS=:systemd:
E: CURRENT_TAGS=:systemd:

P: /devices/pci0000:00/0000:00:1c.0/0000:01:00.1/net/enp1s0f1/ixgbe-mdio-0000:01:00.1
M: ixgbe-mdio-0000:01:00.1
R: 1
U: mdio_bus
E: DEVPATH=/devices/pci0000:00/0000:00:1c.0/0000:01:00.1/net/enp1s0f1/ixgbe-mdio-0000:01:00.1
E: SUBSYSTEM=mdio_bus
...

Important information like: SUBSYSTEM=net, INTERFACE=enp1s0f1, ID_NET_DRIVER=ixgbe, ID_NET_NAME=enp1s0f1 …

Assign 8 Virtual Network Cards:

1
2
root@sense:~# echo 8 > /sys/class/net/enp1s0f0/device/sriov_numvfs 
root@sense:~# echo 8 > /sys/class/net/enp1s0f1/device/sriov_numvfs

Verify network cards again:

1
2
3
4
5
6
7
8
9
10
11
root@sense:~# lspci | grep Ethernet
01:00.0 Ethernet controller: Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
01:00.1 Ethernet controller: Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
01:10.0 Ethernet controller: Intel Corporation 82599 Ethernet Controller Virtual Function (rev 01)
...
01:10.7 Ethernet controller: Intel Corporation 82599 Ethernet Controller Virtual Function (rev 01)
01:11.0 Ethernet controller: Intel Corporation 82599 Ethernet Controller Virtual Function (rev 01)
...
01:11.7 Ethernet controller: Intel Corporation 82599 Ethernet Controller Virtual Function (rev 01)
02:00.0 Ethernet controller: Intel Corporation Ethernet Controller I226-V (rev 04)
03:00.0 Ethernet controller: Intel Corporation Ethernet Controller I226-V (rev 04)

Persist number of SR_IOV Virtual Functions after Proxmox reboot:

1
2
3
root@sense:~# cat /etc/udev/rules.d/enp.rules 
ACTION=="add", SUBSYSTEM=="net", ENV{INTERFACE}=="enp1s0f0", ATTR{device/sriov_numvfs}="8"
ACTION=="add", SUBSYSTEM=="net", ENV{INTERFACE}=="enp1s0f1", ATTR{device/sriov_numvfs}="8"

Add Mapped PCI Devices into Virtual Machine in Proxmox:

Intel Network - Mapped PCI Devices

Virtual Network Cards can be set up as Network Cards, with hardware direct access, from a Network Cards pool (shared by other Virtual Machines), when the first Network Card is available.

Intel Network - Virtual Network Cards

Virtual Machine’s Proxmox configuration file:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
root@sense:/etc/pve/qemu-server# cat 101.conf
agent: 1
bios: ovmf
boot: order=scsi0;ide2;net0
cores: 8
cpu: x86-64-v2-AES,flags=+aes
efidisk0: local-lvm:vm-101-disk-0,efitype=4m,size=4M
hostpci0: mapping=enp0_vf
hostpci1: mapping=enp1_vf
ide2: local:iso/OPNsense-24.7-dvd-amd64.iso,media=cdrom,size=2131548K
machine: q35
memory: 8192
meta: creation-qemu=9.0.2,ctime=1731054484
name: LINKsense
net0: virtio=BC:24:11:0C:52:4B,bridge=vmbr2,firewall=1,queues=4
net1: virtio=BC:24:11:21:3C:E0,bridge=vmbr3,firewall=1,queues=4
numa: 0
ostype: l26
scsi0: local-lvm:vm-101-disk-1,discard=on,iothread=1,size=128G,ssd=1
scsihw: virtio-scsi-single
smbios1: uuid=3e913d90-c284-47d7-881a-599013fa21a4
sockets: 1
vmgenid: aab228d8-bccc-46c1-b627-927271ad20b6

References