From Newsgroup: muc.lists.freebsd.stable

For ${reasons}, I find myself debugging bootup on my server fleet
today. If things are working properly, they only get rebooted a
couple of times a year, so I don't normally see the regular boot
messages that much, but as I'm in the middle of an upgrade cycle, I
started to get annoyed enough to look more closely.

Bug 292604 was the first thing to bother me, and I was happy to see
someone else had reported it, although a bit unhappy that it seems to
have gone ignored and unfixed in 14.4 and presumably 15.1 as well.
It's just an obvious logic error.

I've been trying to find the right way to keep unconfigured network
interfaces from being touched at boot. I initially thought that
setting ifconfig_DEFAULT="NOAUTO" would do the trick, but that also
suppresses configuration of the loopback interface, on which lots of
other services depend, and so the machine boots into an unusable
state. While looking at this, I noticed another issue:

/etc/rc: DEBUG: run_rc_command: doit: routing_start doall
route: message indicates error: File exists
add host 127.0.0.1: gateway lo0 fib 0: route already in table
[a few unrelated debug logs later]
route: message indicates error: File exists
add host ::1: gateway lo0 fib 0: route already in table

I am not at all sure why `routing_start` is doing this, given that the
routes are created automatically when the addresses are configured on lo0. Looking at the script it seems to have something to do with FIBs but
that's not functionality I've needed or touched so I don't know why
it's trying to re-add a route that already exists.

I guess the moral of the story is that even developers don't pay much
attention to boot-time warning messages. Meanwhile, I have to get
back to work.

-GAWollman

PS: annotated boot log of my test system, with debugging turned back off:

Dual Console: Serial Primary, Video Secondary
Setting hostuuid: 85d288de-e87a-0010-84c2-60eb69fe48de.
Setting hostid: 0xfa31a553.
Starting file system checks:
Mounting local filesystems:.
Autoloading module: acpi_wmi
Autoloading module: if_ixl
Autoloading module: pchtherm
acpi_wmi0: <ACPI-WMI mapping> on acpi0
acpi_wmi0: cannot find EC device
[I do not have a single machine on which this works and I don't know
why we have it.]
pchtherm0: <Lewisburg Thermal Subsystem> mem 0x383ffff33000-0x383ffff33fff irq 18 at device 20.2 numa-domain 0 on pci0
ixl0: <Intel(R) Ethernet Connection X722 for 1GbE - 2.3.3-k> mem 0xa9000000-0xa9ffffff,0xaa008000-0xaa00ffff irq 38 at device 0.0 numa-domain 0 on pci7
ixl0: fw 3.1.52349 api 1.5 nvm 3.25 etid 800009e7 oem 1.262.0
ixl0: PF-ID[0]: VFs 32, MSI-X 129, VF MSI-X 5, QPs 768, MDIO shared
ixl0: Using 1024 TX descriptors and 1024 RX descriptors
ixl0: Using 10 RX queues 10 TX queues
ixl0: Using MSI-X interrupts with 11 vectors
ixl0: Ethernet address: ac:1f:6b:65:c7:76
ixl0: Allocating 16 queues for PF LAN VSI; 10 queues active
ixl0: SR-IOV ready
ixl1: <Intel(R) Ethernet Connection X722 for 1GbE - 2.3.3-k> mem 0xa8000000-0xa8ffffff,0xaa000000-0xaa007fff irq 38 at device 0.1 numa-domain 0 on pci7
ixl1: fw 3.1.52349 api 1.5 nvm 3.25 etid 800009e7 oem 1.262.0
ixl1: PF-ID[1]: VFs 32, MSI-X 129, VF MSI-X 5, QPs 768, MDIO shared
ixl1: Using 1024 TX descriptors and 1024 RX descriptors
ixl1: Using 10 RX queues 10 TX queues
ixl1: Using MSI-X interrupts with 11 vectors
ixl1: Ethernet address: ac:1f:6b:65:c7:77
ixl1: Allocating 16 queues for PF LAN VSI; 10 queues active
ixl1: SR-IOV ready
Updating CPU Microcode...
CPU: Intel(R) Xeon(R) Silver 4114 CPU @ 2.20GHz (2200.00-MHz K8-class CPU)
Origin="GenuineIntel" Id=0x50654 Family=0x6 Model=0x55 Stepping=4
Features=0xbfebfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE>
Features2=0x7ffefbff<SSE3,PCLMULQDQ,DTES64,MON,DS_CPL,VMX,SMX,EST,TM2,SSSE3,SDBG,FMA,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,TSCDLT,AESNI,XSAVE,OSXSAVE,AVX,F16C,RDRAND>
AMD Features=0x2c100800<SYSCALL,NX,Page1GB,RDTSCP,LM>
AMD Features2=0x121<LAHF,ABM,Prefetch>
Structured Extended Features=0xd39ffffb<FSGSBASE,TSCADJ,BMI1,HLE,AVX2,FDPEXC,SMEP,BMI2,ERMS,INVPCID,RTM,PQM,NFPUSG,MPX,PQE,AVX512F,AVX512DQ,RDSEED,ADX,SMAP,CLFLUSHOPT,CLWB,PROCTRACE,AVX512CD,AVX512BW,AVX512VL>
Structured Extended Features2=0x18<PKU,OSPKE>
Structured Extended Features3=0xbc002400<MD_CLEAR,TSXFA,IBPB,STIBP,L1DFL,ARCH_CAP,SSBD>
XSAVE Features=0xf<XSAVEOPT,XSAVEC,XINUSE,XSAVES>
IA32_ARCH_CAPS=0x2000c04<RSBA>
VT-x: PAT,HLT,MTF,PAUSE,EPT,UG,VPID,VID,PostIntr
TSC: P-state invariant, performance statistics
Done.
Setting hostname: [REDACTED].
Setting up harvesting: PURE_RDRAND,[CALLOUT],[UMA],[FS_ATIME],SWI,INTERRUPT,NET_NG,[NET_ETHER],NET_TUN,MOUSE,KEYBOARD,ATTACH,CACHED
Feeding entropy: .
ELF ldconfig path: /lib /usr/lib /usr/lib/compat /usr/local/lib /usr/local/lib/compat/pkg /usr/local/lib/compat/pkg /usr/local/lib/perl5/5.42/mach/CORE
32-bit compatibility ldconfig path:
Created clone interfaces: lagg0.
lo0: link state changed to UP
ix0: link state changed to UP
Starting Network: lo0 ix0 ix1 ixl0 ixl1 lagg0.
lo0: flags=1008049<UP,LOOPBACK,RUNNING,MULTICAST,LOWER_UP> metric 0 mtu 16384
options=680003<RXCSUM,TXCSUM,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
inet 127.0.0.1 netmask 0xff000000
inet6 ::1 prefixlen 128
inet6 fe80::1%lo0 prefixlen 64 scopeid 0x3
groups: lo
nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
ix0: flags=1008843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST,LOWER_UP> metric 0 mtu 9120
options=4e53fbb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,WOL_UCAST,WOL_MCAST,WOL_MAGIC,VLAN_HWFILTER,VLAN_HWTSO,RXCSUM_IPV6,TXCSUM_IPV6,HWSTATS,MEXTPG>
ether f8:f2:1e:29:e1:7c
media: Ethernet autoselect (10Gbase-Twinax <full-duplex,rxpause,txpause>)
status: active
nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
ix1: flags=1008843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST,LOWER_UP> metric 0 mtu 9120
options=4e53fbb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,WOL_UCAST,WOL_MCAST,WOL_MAGIC,VLAN_HWFILTER,VLAN_HWTSO,RXCSUM_IPV6,TXCSUM_IPV6,HWSTATS,MEXTPG>
ether f8:f2:1e:29:e1:7c
hwaddr f8:f2:1e:29:e1:7d
media: Ethernet autoselect
status: no carrier
nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
ixl0: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=4e507bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,VLAN_HWFILTER,VLAN_HWTSO,RXCSUM_IPV6,TXCSUM_IPV6,HWSTATS,MEXTPG>
ether ac:1f:6b:65:c7:76
media: Ethernet autoselect
status: no carrier
nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
ixl1: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=4e507bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,VLAN_HWFILTER,VLAN_HWTSO,RXCSUM_IPV6,TXCSUM_IPV6,HWSTATS,MEXTPG>
ether ac:1f:6b:65:c7:77
media: Ethernet autoselect
status: no carrier
nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
[these ixl(4) interfaces are not connected to anything and do not
need to be initialized]
lagg0: flags=1008843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST,LOWER_UP> metric 0 mtu 9120
options=4e53fbb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,WOL_UCAST,WOL_MCAST,WOL_MAGIC,VLAN_HWFILTER,VLAN_HWTSO,RXCSUM_IPV6,TXCSUM_IPV6,HWSTATS,MEXTPG>
ether f8:f2:1e:29:e1:7c
hwaddr 00:00:00:00:00:00
inet [redacted] netmask 0xffffff00 broadcast [redacted]
laggproto lacp lagghash l2,l3,l4
laggport: ix0 flags=0<>
laggport: ix1 flags=0<>
groups: lagg
media: Ethernet autoselect
status: active
nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
Starting devd.
ix1: link state changed to UP
lagg0: link state changed to UP
syscons does not exist in /etc/rc.d or the local startup
directories (/usr/local/etc/rc.d), or is not executable
moused does not exist in /etc/rc.d or the local startup
directories (/usr/local/etc/rc.d), or is not executable
[no idea where these are getting called from, maybe devd?]
Starting Network: ixl0.
ixl0: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=4e507bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,VLAN_HWFILTER,VLAN_HWTSO,RXCSUM_IPV6,TXCSUM_IPV6,HWSTATS,MEXTPG>
ether ac:1f:6b:65:c7:76
media: Ethernet autoselect
status: no carrier
nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
[once again, no need to touch this]
Starting Network: ixl1.
ixl1: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=4e507bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,VLAN_HWFILTER,VLAN_HWTSO,RXCSUM_IPV6,TXCSUM_IPV6,HWSTATS,MEXTPG>
ether ac:1f:6b:65:c7:77
media: Ethernet autoselect
status: no carrier
nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
[or this]
Starting zfsd.
route: message indicates error: File exists
add host 127.0.0.1: gateway lo0 fib 0: route already in table
add net default: gateway [redacted]
route: message indicates error: File exists
add host ::1: gateway lo0 fib 0: route already in table
add net fe80::: gateway ::1
add net ff02::: gateway ::1
add net ::ffff:0.0.0.0: gateway ::1
add net ::0.0.0.0: gateway ::1
Flushed all rules.
00100 allow ip from any to any via lo0
00200 deny ip from any to 127.0.0.0/8
00300 deny ip from 127.0.0.0/8 to any
00400 deny ip from any to ::1
00500 deny ip from ::1 to any
00600 allow ipv6-icmp from :: to ff02::/16
00700 allow ipv6-icmp from fe80::/10 to fe80::/10
00800 allow ipv6-icmp from fe80::/10 to ff02::/16
00900 allow ipv6-icmp from any to any icmp6types 1
01000 allow ipv6-icmp from any to any icmp6types 2,135,136
Line 1: failed to flush table 1 info: No such process
Line 2: DEPRECATED: inserting data into non-existent table 1. (auto-created) added: [multiple addresses redacted] 0
Line 8: failed to flush table 2 info: No such process
Line 9: DEPRECATED: inserting data into non-existent table 2. (auto-created) added: [multiple IPv6 addresses redacted] 0
01010 allow tcp from table(1) to me 111
01011 allow tcp from table(1) to me 2049
01012 allow udp from table(1) to me 111
01013 allow udp from table(1) to me 2049
01020 allow ip6 from table(2) to me6 111 proto tcp
01021 allow ip6 from table(2) to me6 2049 proto tcp
01022 allow ip6 from table(2) to me6 111 proto udp
01023 allow ip6 from table(2) to me6 2049 proto udp
01040 deny tcp from any to me 111
01041 deny tcp from any to me 2049
01042 deny udp from any to me 111
01043 deny udp from any to me 2049
01044 deny ip6 from any to me6 111 proto tcp
01045 deny ip6 from any to me6 2049 proto tcp
01046 deny ip6 from any to me6 111 proto udp
01047 deny ip6 from any to me6 2049 proto udp
02000 allow tcp from table(1) to me 22
02001 allow ip6 from table(2) to me6 22 proto tcp
02002 allow tcp from [redacted] to me 22
02003 allow ip6 from [redacted] to me6 22 proto tcp
02010 deny tcp from any to me 22
02011 deny ip6 from any to me6 22 proto tcp
64999 deny ip from table(22) to any
Firewall rules loaded.
Waiting for [redacted] to respond to ICMP ping, got response.
[this is after fixing the bug in netwait]
Obtaining a trust anchor...
Starting unbound.
Starting nfsuserd.
Starting nslcd.
Clearing /tmp (X related).
Updating /var/run/os-release done.
Recovering vi editor sessions:.
Creating and/or trimming log files.
Updating motd:.
Starting syslogd.
Starting auditd.
Starting rpcbind.
Security policy loaded: MAC/ntpd (mac_ntpd)
Starting ntpd.
Mounting late filesystems:.
Starting munin_node.
Performing sanity check on openssh configuration.
Starting openssh.
NFSv4 only server
Starting mountd.
Starting lldpd.
Starting sshguard.
Starting sendmail_submit.
Starting sendmail_msp_queue.
Starting puppet.
Starting nrpe.
Starting cron.
Starting nfsd.

Starting background file system checks in 60 seconds.
Starting smartd.

[it's maybe worth noting some of the inconsistencies.]


--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-admin@muc.de
--- Synchronet 3.22a-Linux NewsLink 1.2