_______ ________ __ | |.-----.-----.-----.| | | |.----.| |_ | - || _ | -__| || | | || _|| _| |_______|| __|_____|__|__||________||__| |____| |__| W I R E L E S S F R E E D O M ----------------------------------------------------- OpenWrt 24.10.0-rc2, r28161-ea17e958b9 ----------------------------------------------------- root@SenseWrt:~# tailscale up --netfilter-mode=off --advertise-routes=192.168.88.0/24 --accept-routes Warning: netfilter=off; configure iptables yourself. Warning: UDP GRO forwarding is suboptimally configured on eth1, UDP forwarding throughput capability will increase with a configuration change. See https://tailscale.com/s/ethtool-config-udp-gro
To authenticate, visit:
https://login.tailscale.com/a/98c452901c4ba
Success.
NOTE:192.168.88.0/24 is the ip range of local network setup in OpenWrt.
Disable key expiry for OpenWrt machine in Tailscale console, then enable all OpenWrt clients access Tailscale network:
Now add Tailscale virtual network as a new interface in OpenWrt:
Create firewall for Tailscale virtual network interface in OpenWrt:
Configure firewall for Tailscale virtual network interface in OpenWrt:
NOTE:opt network is for the downstream DHCP clients.
_______ ________ __ | |.-----.-----.-----.| | | |.----.| |_ | - || _ | -__| || | | || _|| _| |_______|| __|_____|__|__||________||__| |____| |__| W I R E L E S S F R E E D O M ----------------------------------------------------- OpenWrt 24.10.0-rc2, r28161-ea17e958b9 -----------------------------------------------------
root@SenseWrt:~# cat /etc/config/zerotier
config zerotier 'global' # Sets whether ZeroTier is enabled or not option enabled 1 # Sets the ZeroTier listening port (default 9993; set to 0 for random) #option port '9993' # Client secret (leave blank to generate a secret on first run) option secret '' # Path of the optional file local.conf (see documentation at # https://docs.zerotier.com/config#local-configuration-options) #option local_conf_path '/etc/zerotier.conf' # Persistent configuration directory (to perform other configurations such # as controller mode or moons, etc.) #option config_path '/etc/zerotier' # Copy the contents of the persistent configuration directory to memory # instead of linking it, this avoids writing to flash #option copy_config_path '1'
# Network configuration, you can have as many configurations as networks you # want to join (the network name is optional) config network 'earth' # Identifier of the network you wish to join option id '8ca917257083e297' # Network configuration parameters (all are optional, if not indicated the # default values are set, see documentation at # https://docs.zerotier.com/config/#network-specific-configuration) option allow_managed '1' option allow_global '0' option allow_default '0' option allow_dns '0'
# Example of a second network (unnamed as it is optional) #config network # option id '1234567890123456' # option allow_managed '1' # option allow_global '0' # option allow_default '0' # option allow_dns '0'
_______ ________ __ | |.-----.-----.-----.| | | |.----.| |_ | - || _ | -__| || | | || _|| _| |_______|| __|_____|__|__||________||__| |____| |__| W I R E L E S S F R E E D O M ----------------------------------------------------- OpenWrt 23.05.5, r24106-10cc5fcd00 -----------------------------------------------------
# increase the raw disk root@sense:~# qemu-img resize -f raw ./openwrt.raw 1024M Image resized.
Convert OpenWRT image to VM disk:
1 2 3 4 5 6 7 8 9
# import the raw disk to OpenWRT VM root@sense:~# qm importdisk 104 openwrt.raw local-lvm Use of uninitialized value $dev in hash element at /usr/share/perl5/PVE/QemuServer/Drive.pm line 555. importing disk 'openwrt.raw' to VM 104 ... Logical volume "vm-104-disk-0" created. transferred 0.0 B of 1.0 GiB (0.00%) ... transferred 1.0 GiB of 1.0 GiB (100.00%) unused0: successfully imported disk 'local-lvm:vm-104-disk-0'
Double click the Unused Disk, then click the Add button:
Configure OpenWRT
Start up VM; change the user root password; set LAN ip address temporarily to 192.168.2.3 (Default: 192.168.1.1):
$ ssh -l root 192.168.2.3 The authenticity of host '192.168.2.3 (192.168.2.3)' can't be established. ED25519 key fingerprint is SHA256:AggWAL1oU8+r1f84KoqpvcsYUylZOTfN0sXwHSby3b0. This key is not known by any other names. Are you sure you want to continue connecting (yes/no/[fingerprint])? yes Warning: Permanently added '192.168.2.3' (ED25519) to the list of known hosts. root@192.168.2.3's password: _______ ________ __ | |.-----.-----.-----.| | | |.----.| |_ | - || _ | -__| || | | || _|| _| |_______|| __|_____|__|__||________||__| |____| |__| W I R E L E S S F R E E D O M ----------------------------------------------------- OpenWrt 23.05.5, r24106-10cc5fcd00 -----------------------------------------------------
root@OpenWrt:~# ls -al /var/opkg-lists/ drwxr-xr-x 2 root root 320 Dec 6 03:15 . drwxrwxrwt 17 root root 440 Dec 6 03:15 .. -rw-r--r-- 1 root root 72705 Dec 6 03:15 openwrt_base -rw-r--r-- 1 root root 142 Dec 6 03:15 openwrt_base.sig -rw-r--r-- 1 root root 10039 Dec 6 03:15 openwrt_core -rw-r--r-- 1 root root 142 Dec 6 03:15 openwrt_core.sig -rw-r--r-- 1 root root 100416 Dec 6 03:15 openwrt_kmods -rw-r--r-- 1 root root 142 Dec 6 03:15 openwrt_kmods.sig -rw-r--r-- 1 root root 197242 Dec 6 03:15 openwrt_luci -rw-r--r-- 1 root root 142 Dec 6 03:15 openwrt_luci.sig -rw-r--r-- 1 root root 516243 Dec 6 03:15 openwrt_packages -rw-r--r-- 1 root root 142 Dec 6 03:15 openwrt_packages.sig -rw-r--r-- 1 root root 9918 Dec 6 03:15 openwrt_routing -rw-r--r-- 1 root root 142 Dec 6 03:15 openwrt_routing.sig -rw-r--r-- 1 root root 58240 Dec 6 03:15 openwrt_telephony -rw-r--r-- 1 root root 142 Dec 6 03:15 openwrt_telephony.sig
There is some issue with IPv6 support in OpenWRT when download update. Errors thrown when wan connected to a IPv6 router:
1 2 3 4 5 6 7 8 9 10 11 12 13
root@OpenWrt:~# opkg update Downloading https://downloads.openwrt.org/releases/23.05.5/targets/x86/64/packages/Packages.gz *** Failed to download the package list from https://downloads.openwrt.org/releases/23.05.5/targets/x86/64/packages/Packages.gz ... Downloading https://downloads.openwrt.org/releases/23.05.5/packages/x86_64/telephony/Packages.gz *** Failed to download the package list from https://downloads.openwrt.org/releases/23.05.5/packages/x86_64/telephony/Packages.gz
Collected errors: * opkg_download: Failed to download https://downloads.openwrt.org/releases/23.05.5/targets/x86/64/packages/Packages.gz, wget returned 4. * opkg_download: Check your network settings and connectivity. ... * opkg_download: Failed to download https://downloads.openwrt.org/releases/23.05.5/packages/x86_64/telephony/Packages.gz, wget returned 4. * opkg_download: Check your network settings and connectivity.
Turn off IPv6 on router to workaround.
1 2 3 4 5 6 7 8 9 10
root@OpenWRT:~# opkg update Downloading https://downloads.openwrt.org/releases/23.05.5/targets/x86/64/packages/Packages.gz Updated list of available packages in /var/opkg-lists/openwrt_core Downloading https://downloads.openwrt.org/releases/23.05.5/targets/x86/64/packages/Packages.sig Signature check passed. ... Downloading https://downloads.openwrt.org/releases/23.05.5/packages/x86_64/telephony/Packages.gz Updated list of available packages in /var/opkg-lists/openwrt_telephony Downloading https://downloads.openwrt.org/releases/23.05.5/packages/x86_64/telephony/Packages.sig Signature check passed.
Need this handy tool to find all the missing parts:
Driver Identifier, a digital assistant for your system’s hardware, can scan the device, identifies outdated or missing drivers, and provides a customized list of updates for your specific hardware:
References
DriverIdentifier is a powerful tool that takes the hassle out of managing and updating drivers on your computer, https://www.driveridentifier.com/
$ cat /etc/resolv.conf # # macOS Notice # # This file is not consulted for DNS hostname resolution, address # resolution, or the DNS query routing mechanism used by most # processes on this system. # # To view the DNS configuration used by this system, use: # scutil --dns # # SEE ALSO # dns-sd(1), scutil(8) # # This file is automatically generated. # search local nameserver 2403:5802:8c44:3:be24:11ff:fe21:3ce0 nameserver 192.168.68.1
root@sense:~# lspci -v ... 01:00.0 Ethernet controller: Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01) Flags: bus master, fast devsel, latency 0, IRQ 16, IOMMU group 12 Memory at 80a20000 (64-bit, non-prefetchable) [size=128K] I/O ports at 3020 [disabled] [size=32] Memory at 80a44000 (64-bit, non-prefetchable) [size=16K] Capabilities: [40] Power Management version 3 Capabilities: [50] MSI: Enable- Count=1/1 Maskable+ 64bit+ Capabilities: [70] MSI-X: Enable+ Count=64 Masked- Capabilities: [a0] Express Endpoint, MSI 00 Capabilities: [e0] Vital Product Data Capabilities: [100] Advanced Error Reporting Capabilities: [140] Device Serial Number a8-b8-e0-ff-ff-05-96-4d Capabilities: [150] Alternative Routing-ID Interpretation (ARI) Capabilities: [160] Single Root I/O Virtualization (SR-IOV) Kernel driver in use: ixgbe Kernel modules: ixgbe
01:00.1 Ethernet controller: Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01) Flags: bus master, fast devsel, latency 0, IRQ 17, IOMMU group 13 Memory at 80a00000 (64-bit, non-prefetchable) [size=128K] I/O ports at 3000 [disabled] [size=32] Memory at 80a40000 (64-bit, non-prefetchable) [size=16K] Capabilities: [40] Power Management version 3 Capabilities: [50] MSI: Enable- Count=1/1 Maskable+ 64bit+ Capabilities: [70] MSI-X: Enable+ Count=64 Masked- Capabilities: [a0] Express Endpoint, MSI 00 Capabilities: [e0] Vital Product Data Capabilities: [100] Advanced Error Reporting Capabilities: [140] Device Serial Number a8-b8-e0-ff-ff-05-96-4d Capabilities: [150] Alternative Routing-ID Interpretation (ARI) Capabilities: [160] Single Root I/O Virtualization (SR-IOV) Kernel driver in use: ixgbe Kernel modules: ixgbe
01:10.0 Ethernet controller: Intel Corporation 82599 Ethernet Controller Virtual Function (rev 01) Flags: bus master, fast devsel, latency 0, IOMMU group 18 Memory at 4017000000 (64-bit, prefetchable) [virtual] [size=16K] Memory at 4017100000 (64-bit, prefetchable) [virtual] [size=16K] Capabilities: [70] MSI-X: Enable+ Count=3 Masked- Capabilities: [a0] Express Endpoint, MSI 00 Capabilities: [100] Advanced Error Reporting Capabilities: [150] Alternative Routing-ID Interpretation (ARI) Kernel driver in use: vfio-pci Kernel modules: ixgbevf
...
02:00.0 Ethernet controller: Intel Corporation Ethernet Controller I226-V (rev 04) Subsystem: Intel Corporation Ethernet Controller I226-V Flags: bus master, fast devsel, latency 0, IRQ 18, IOMMU group 14 Memory at 80600000 (32-bit, non-prefetchable) [size=1M] Memory at 80700000 (32-bit, non-prefetchable) [size=16K] Capabilities: [40] Power Management version 3 Capabilities: [50] MSI: Enable- Count=1/1 Maskable+ 64bit+ Capabilities: [70] MSI-X: Enable+ Count=5 Masked- Capabilities: [a0] Express Endpoint, MSI 00 Capabilities: [100] Advanced Error Reporting Capabilities: [140] Device Serial Number a8-b8-e0-ff-ff-05-96-4f Capabilities: [1c0] Latency Tolerance Reporting Capabilities: [1f0] Precision Time Measurement Capabilities: [1e0] L1 PM Substates Kernel driver in use: igc Kernel modules: igc
03:00.0 Ethernet controller: Intel Corporation Ethernet Controller I226-V (rev 04) Subsystem: Intel Corporation Ethernet Controller I226-V Flags: bus master, fast devsel, latency 0, IRQ 16, IOMMU group 15 Memory at 80400000 (32-bit, non-prefetchable) [size=1M] Memory at 80500000 (32-bit, non-prefetchable) [size=16K] Capabilities: [40] Power Management version 3 Capabilities: [50] MSI: Enable- Count=1/1 Maskable+ 64bit+ Capabilities: [70] MSI-X: Enable+ Count=5 Masked- Capabilities: [a0] Express Endpoint, MSI 00 Capabilities: [100] Advanced Error Reporting Capabilities: [140] Device Serial Number a8-b8-e0-ff-ff-05-96-50 Capabilities: [1c0] Latency Tolerance Reporting Capabilities: [1f0] Precision Time Measurement Capabilities: [1e0] L1 PM Substates Kernel driver in use: igc Kernel modules: igc
04:00.0 Network controller: Intel Corporation Wi-Fi 7(802.11be) AX1775*/AX1790*/BE20*/BE401/BE1750* 2x2 (rev 1a) Subsystem: Intel Corporation Wi-Fi 7(802.11be) AX1775*/AX1790*/BE20*/BE401/BE1750* 2x2 (BE200 320MHz [Gale Peak]) Flags: bus master, fast devsel, latency 0, IRQ 17, IOMMU group 16 Memory at 80900000 (64-bit, non-prefetchable) [size=16K] Capabilities: [40] Power Management version 3 Capabilities: [50] MSI: Enable- Count=1/1 Maskable- 64bit+ Capabilities: [70] Express Endpoint, MSI 00 Capabilities: [b0] MSI-X: Enable+ Count=32 Masked- Capabilities: [100] Advanced Error Reporting Capabilities: [148] Secondary PCI Express Capabilities: [158] Physical Layer 16.0 GT/s <?> Capabilities: [17c] Lane Margining at the Receiver <?> Capabilities: [188] Latency Tolerance Reporting Capabilities: [190] L1 PM Substates Capabilities: [1a0] Vendor Specific Information: ID=0002 Rev=4 Len=100 <?> Capabilities: [2a0] Data Link Feature <?> Capabilities: [2ac] Precision Time Measurement Capabilities: [2b8] Vendor Specific Information: ID=0003 Rev=1 Len=054 <?> Capabilities: [500] Vendor Specific Information: ID=0023 Rev=1 Len=010 <?> Kernel driver in use: iwlwifi Kernel modules: iwlwifi ...
root@sense:~# ip a ... 2: enp2s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master vmbr2 state UP group default qlen 1000 link/ether a8:b8:e0:05:96:4f brd ff:ff:ff:ff:ff:ff 3: enp3s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master vmbr3 state UP group default qlen 1000 link/ether a8:b8:e0:05:96:50 brd ff:ff:ff:ff:ff:ff 4: enp1s0f0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master vmbr0 state UP group default qlen 1000 link/ether a8:b8:e0:05:96:4d brd ff:ff:ff:ff:ff:ff 5: enp1s0f1: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq master vmbr1 state DOWN group default qlen 1000 link/ether a8:b8:e0:05:96:4e brd ff:ff:ff:ff:ff:ff 7: enp1s0f0v1: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000 link/ether 7a:de:19:c7:c2:ce brd ff:ff:ff:ff:ff:ff ... 22: wlp4s0f0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000 link/ether e0:8f:4c:b2:58:95 brd ff:ff:ff:ff:ff:ff
IOMMU PT mode improves the performance of other PCIe devices in the system when passthrough is being used.
Update GRUB:
1 2 3 4 5 6 7 8 9
root@sense:~# update-grub Generating grub configuration file ... Found linux image: /boot/vmlinuz-6.8.12-2-pve Found initrd image: /boot/initrd.img-6.8.12-2-pve Found linux image: /boot/vmlinuz-6.8.4-2-pve Found initrd image: /boot/initrd.img-6.8.4-2-pve Found memtest86+ 64bit EFI image: /boot/memtest86+x64.efi Adding boot menu entry for UEFI Firmware Settings ... done
root@sense:~# dmesg | grep -i IOMMU [ 0.000000] Command line: BOOT_IMAGE=/boot/vmlinuz-6.8.12-2-pve root=/dev/mapper/pve-root ro quiet intel_iommu=on iommu=pt [ 0.053988] Kernel command line: BOOT_IMAGE=/boot/vmlinuz-6.8.12-2-pve root=/dev/mapper/pve-root ro quiet intel_iommu=on iommu=pt [ 0.054043] DMAR: IOMMU enabled [ 0.145258] DMAR-IR: IOAPIC id 2 under DRHD base 0xfed91000 IOMMU 1 [ 0.403235] pci 0000:00:02.0: DMAR: Skip IOMMU disabling for graphics [ 0.440538] iommu: Default domain type: Translated [ 0.440538] iommu: DMA domain TLB invalidation policy: lazy mode [ 0.483908] DMAR: IOMMU feature fl1gp_support inconsistent [ 0.483909] DMAR: IOMMU feature pgsel_inv inconsistent [ 0.483910] DMAR: IOMMU feature nwfs inconsistent [ 0.483911] DMAR: IOMMU feature dit inconsistent [ 0.483912] DMAR: IOMMU feature sc_support inconsistent [ 0.483912] DMAR: IOMMU feature dev_iotlb_support inconsistent [ 0.493686] pci 0000:00:02.0: Adding to iommu group 0 [ 0.493726] pci 0000:00:00.0: Adding to iommu group 1 [ 0.493739] pci 0000:00:0d.0: Adding to iommu group 2 [ 0.493764] pci 0000:00:14.0: Adding to iommu group 3 [ 0.493774] pci 0000:00:14.2: Adding to iommu group 3 [ 0.493786] pci 0000:00:16.0: Adding to iommu group 4 [ 0.493793] pci 0000:00:17.0: Adding to iommu group 5 [ 0.493806] pci 0000:00:1c.0: Adding to iommu group 6 [ 0.493820] pci 0000:00:1c.6: Adding to iommu group 7 [ 0.493831] pci 0000:00:1d.0: Adding to iommu group 8 [ 0.493842] pci 0000:00:1d.1: Adding to iommu group 9 [ 0.493859] pci 0000:00:1d.3: Adding to iommu group 10 [ 0.493878] pci 0000:00:1f.0: Adding to iommu group 11 [ 0.493886] pci 0000:00:1f.3: Adding to iommu group 11 [ 0.493895] pci 0000:00:1f.4: Adding to iommu group 11 [ 0.493903] pci 0000:00:1f.5: Adding to iommu group 11 [ 0.493918] pci 0000:01:00.0: Adding to iommu group 12 [ 0.493934] pci 0000:01:00.1: Adding to iommu group 13 [ 0.493945] pci 0000:02:00.0: Adding to iommu group 14 [ 0.493956] pci 0000:03:00.0: Adding to iommu group 15 [ 0.493982] pci 0000:04:00.0: Adding to iommu group 16 [ 0.493993] pci 0000:05:00.0: Adding to iommu group 17 [ 3.684070] pci 0000:01:10.1: Adding to iommu group 18 [ 3.684303] pci 0000:01:10.3: Adding to iommu group 19 [ 3.684501] pci 0000:01:10.5: Adding to iommu group 20 [ 3.684685] pci 0000:01:10.7: Adding to iommu group 21 [ 3.684885] pci 0000:01:11.1: Adding to iommu group 22 [ 3.685096] pci 0000:01:11.3: Adding to iommu group 23 [ 3.685296] pci 0000:01:11.5: Adding to iommu group 24 [ 3.685473] pci 0000:01:11.7: Adding to iommu group 25 [ 3.755233] pci 0000:01:10.0: Adding to iommu group 26 [ 3.755462] pci 0000:01:10.2: Adding to iommu group 27 [ 3.755761] pci 0000:01:10.4: Adding to iommu group 28 [ 3.755939] pci 0000:01:10.6: Adding to iommu group 29 [ 3.756107] pci 0000:01:11.0: Adding to iommu group 30 [ 3.756276] pci 0000:01:11.2: Adding to iommu group 31 [ 3.756444] pci 0000:01:11.4: Adding to iommu group 32 [ 3.756628] pci 0000:01:11.6: Adding to iommu group 33
ATTENTION: IOMMU Group not order by PCI BUS IDs
This is an ISSUE CAN’T setup PCI Device Resource Mappings, because of errors, e.g., “Configuration for iommugroup not correct (‘21’ != ‘’27)” thrown, sometime, after Proxmox reboot!
Work around solution is go to Resource Mappings in Proxmox, edit, make no change and save:
Filter out network card udev information, e.g., enp1s0f1:
Add Mapped PCI Devices into Virtual Machine in Proxmox:
Virtual Network Cards can be set up as Network Cards, with hardware direct access, from a Network Cards pool (shared by other Virtual Machines), when the first Network Card is available.
This is the step by step guide, the fastest and easiest way install and run latest MacOS Sequoia 15.0.1 in VMware Workstation Pro on Windows 11.
Due to MacOS Sequoia has added detection whether OS is running in Virtual Machine, so it’s better install MacOS Sonoma at first, then upgrade to latest version MacOS Sequoia.
M. Change Max-OS Version (Currently 12) C. Change Catalog (Currently publicrelease) I. Only Print URLs (Currently Off) S. Set Current Catalog to SoftwareUpdate Catalog L. Clear SoftwareUpdate Catalog R. Toggle Recovery-Only (Currently Off) U. Show Catalog URL Q. Quit
Please select an option: 3
Downloading InstallAssistant.pkg for 062-78824 - 14.7 macOS Sonoma (23H124)...
DON’T enable Location Service during the installation! Otherwise, you can’t setup Time Zone, Date Time based on your area. You can login with your Apple ID during the installation.
Install VMware Tools
After MacOS installed and VM restarted, mount darwin.iso and install VMware Tools. Then Display Memory in MacOS becomes 128 MB, and support the Full Screen mode.
Upgrade MacOS
After MacOS Sonoma installed, make sure everything is OK, then copy the whole MacOS Sonoma directory to a new directory MacOS Sequoia (Upgraded). Open the new directory in VMware, and select I copied it, then you can upgrade MacOS to latest verion in System Settings -> Software Update.
