Randy Bush efa75d4dc9 migrate to balance cluster with new vm0

2025-10-16 18:07:41 -07:00

18 KiB

Raw Blame History

Ashburn DC2 PoP

Sandbox, cage 2020, rack 102
vm1 and r0 in rack 0103
c/o Equinix DC2
21715 Filigree Ct
Ashburn, VA 20147
noc@equinix.com (or ask ERC support@equinix.com folk to escalate to
                 primary on-call NSE)
+1 888 892 0607 equinix 24x7

CURRENT CONTACTS LIST: (2025.08.07)
  Potcholo Nicolas <pnicolas@equinix.com>  +1 571 302 6293
  Joe Corrado <jcorrado@equinix.com>

2024.07.02
  Muhammad Waqas Bin Tariq <mwaqas@equinix.com>
  Tarkesh Patel <tpatel@equinix.com>

PoP Diagram

Circuits

Cogent

A-side IBX: DC2 Cage: 2020 Patch Panel: PP:0103:25916 Ports: 11/12 (these were new ports)

Cogent
- GigabitEthernet0/2/1/8.rcr22.iad01
- description CUST-W GIGE ^RGnet LLC^ ID:3-001717104 PT:3-001717104
- AS 174
- ipv4 address 38.88.248.236/31
- ipv6 address 2001:550:2:1::22/127
RGnet/PSGnet
- r1.iad-eno1
- AS 3927
- ipv4 address 38.88.248.237/31
- ipv6 address 2001:550:2:1::23/127

NTT

NTT
- AS 2914
- ipv4 address 129.250.10.157/30
- ipv6 address 2001:418:8000:5000::19/126
RGnet/PSGnet
- r0.iad-eno4 (via sw0 as media converter)
- AS 3927
- ipv4 address 129.250.10.158/30
- ipv6 address 2001:418:8000:5000::1a/126

o0.iad

o0 IP into Equinix Management Network
port 20 of Louie's 3Com switch in the next cabinet over
o0.iad.rg.net: 206.223.132.86/26
gateway: 206.223.132.65

Route Views

Circuit ID 20438058 in cage 2020 is Route Views to IX

Equipment

sw0 is a Cisco WS-C3750G-24PS
sw1 is a Cisco Nexus C3064PQ
o0 is a Cisco 2621 (MPC860) processor (revision 0x102) w/ 61440K/4096K
r0 is a Dell 630 running FRR
r1 is a Dell 630 running FRR
Equinix Power Strip aka Leviton
photo from 2020.02.18

Rack Physical and Wiring

hostname	switch	oob	oobname	pwr	RU
r0.iad.rg.net	s0.24 s0.11	o0.33	r0	0.2 1.3		Dell R630 6TKCMN2
r1.iad.rg.net	s0.05 s0.03	o0.36	r1	0.1 1.1	????	Dell 630/FRR
vm0.iad.rg.net	s1.1&2 s0.14	o0.42	vm0	0.6	3	SuperMicro SYS-6016T-URF
vm1.iad.rg.net	s1.3&4 s0.12	o0.37	vm1	0.3 1.4		Dell R630 7WCXDH2
vm2.iad.rg.net	s1.5&6 s0.13	o0.35	vm2	0.4 1.2		Dell R630 GLY2W52
o0.iad.rg.net	s0.01/02	vm2.usb		1.6		Cisco 2621 (MPC860)
sw0.iad.rg.net	s1.48	o0.45	sw0	0.5		Cisco WS-C3750G-24PS
sw1.iad.rg.net	s0.25	o0.48	sw1	floor		Cisco 3964
pwr0.iad.rg.net		o0.41	pwr0	equinix		BayTech RPC
pwr1.iad.rg.net		o0.43	pwr1	equinix		BayTech RPC
fiber patch panel					5-6	(equinix)

Ethernet Ports

hostname	main ether	ipmi	drbd
vm0.iad.rg.net	sw1.01	sw0.14	sw1.02
vm1.iad.rg.net	sw1.03	sw0.12	sw1.04
vm2.iad.rg.net	sw1.05	sw0.13	sw1.06

hostname	main ether	ipmi
r0 .iad.rg.net	sw0.24	sw0.11
r1 .iad.rg.net	sw0.05	sw0.03

RGnet Cluster / 198.180.150.4 198.180.150.10

vm0.iad.rg.net/home/randy# qm list
      VMID NAME                 STATUS     MEM(MB)    BOOTDISK(GB) PID       
       100 tor0.iad.rg.net      running    1024              32.00 62405     
       103 pfs0.iad.rg.net      running    512                8.00 64124     
       104 pfs.iad.rg.net       running    1024               8.00 62070     
       106 srv0.iad.rg.net      running    2048              64.00 63469     
       112 netsrv.iad.rg.net    running    2048              32.00 62956     

vm1.iad.rg.net:/home/randy# qm list
      VMID NAME                 STATUS     MEM(MB)    BOOTDISK(GB) PID       
       101 bbte.rg.net          running    4096              64.00 2501      
       102 bozanicfamily.com    running    1024              34.00 2662      
       105 rgnet.anycast.censurfridns.dk running    6144              50.00 3211      
       107 m2.machshav.com      running    8192             125.00 3551      
       108 ns1.machshav.com     running    2048              32.00 3712      

vm2.iad.rg.net:/root# qm list
      VMID NAME                 STATUS     MEM(MB)    BOOTDISK(GB) PID       
       109 bbgp.rg.net          running    2048              32.00 2596      
       110 bbt2.rg.net          running    4096              64.00 2812      
       111 khatovar.hactrn.net  running    4096             250.00 3019      
       113 usbeacon2.iibex.auckland.ac.nz running    1024              36.00 3450      
       114 wg.iad.rg.net        running    2048              10.00 3738

Hostname	IP Address	Owner and Email
bbgp.rg.net	198.180.150.15	Randy Bush <randy@psg.com>
bbt2.rg.net	198.180.150.61	Cristel Pelsser <cristel.pelsser@uclouvain.be>
bbte.rg.net	198.180.150.60	Cristel Pelsser <cristel.pelsser@uclouvain.be>
bozanicfamily.com	198.180.150.5	Randy Bush <randy@psg.com>
khatovar.hactrn.net	198.180.150.30	Rob Austein <sra@hactrn.net>
m2.macshav.com	198.180.150.45	Steven M Bellovin <smb@cs.columbia.edu>
netsrv.iad.rg.net	198.180.150.13	Randy Bush <randy@psg.com>
ns1.machshav.com	198.180.150.41	Steven M Bellovin <smb@cs.columbia.edu>
pfs0.iad.rg.net	198.180.150.3	Randy Bush <randy@psg.com>
pfs.iad.rg.net	198.180.150.6	Randy Bush <randy@psg.com>
rgnet.anycast.censurfridns.dk	198.180.150.12	Phil Regnauld <pr@x0.dk>
srv0.iad.rg.net	198.180.150.1	Randy Bush <randy@psg.com>
tor0.iad.rg.net	198.180.150.9	Randy Bush <randy@psg.com>
usbeacon2.iibex.auckland.ac.nz	198.180.150.52	Etuate Cocker <ecoc005@aucklanduni.ac.nz>
wg.iad.rg.net	198.180.150.16	Randy Bush <randy@psg.com>

vm0.iad.rg.net proxmox

Dell R630
Service Tag: 7WD2HH2
Express Service Code: 17194344950
2x 12-Core 2.4 GHz Intel E5-2670v3
256GB DDR4 Memory
H730 Mini RAID controller
4 Gigabit RJ45 network ports
2 10Gb SFP+
iDRAC 8 Enterprise
iDRAC MAC: 84:7B:EB:F5:54:48 Intel X520-SR2 2-Port 10G SFP+ Ethernet, PCI-e

vm1.iad.rg.net Disk Drives

Unit	Model	Serial
0	Samsung SSD 870 EVO 4TB	S758NX0X504363Z	new as of 2025.10.16
1	Samsung SSD 870 EVO 4TB	S758NX0X504346A	new as of 2025.10.16
2	Samsung SSD 870 EVO 4TB	S758NX0X504359A	new as of 2025.10.16
3	Samsung SSD 870 EVO 4TB	S758NX0X504345N	new as of 2025.10.16

vm1.iad.rg.net proxmox

Service Tag: 7WCXDH2
Dell R630
2x 10-Core 2.4 GHz Intel E5-2640V4
256GB DDR4 Memory
H730 Mini RAID controller
4 Gigabit RJ45 network ports
2 10Gb SFP+
iDRAC Enterprise
Intel X520-SR2 2-Port 10G SFP+ Ethernet, PCI-e

vm1.iad.rg.net Disk Drives

Unit	Model	Serial
0	Samsung SSD 870 EVO 4TB	S758NX0X504317Z	new as of 2025.07.08
1	Samsung SSD 870 EVO 4TB	S758NX0X505056R	new as of 2025.07.08
2	Samsung SSD 870 EVO 4TB	S758NX0X504319J	new as of 2025.07.08
3	Samsung SSD 870 EVO 4TB	S758NX0X504324X	new as of 2025.07.08

vm2.iad.rg.net Hardware

Dell R630 Tag: GLY2W52 Express Code: 36155548982
Dual Intel(R) Xeon(R) CPU E5-2697 v3 @ 2.60GHz
192GB RAM 12x16GB DDR4
iDRAC 8 Enterprise
3ware Inc 9750 Controller
BRCM GbE 4P 5720-t rNDC
Intel X520-SR2 2-Port 10G SFP+ Ethernet, PCI-e

vm2.iad Disk Drives

for i in a b c d; do echo $i; smartctl -a /dev/sd$i | egrep '^(Device Model|Serial)' | cut -c 19-60; done

LUN	Serial	Model	Size
a	S6PNNS0T602439V	Samsung SSD 870 EVO	2TB
b	S6PNNS0T602496T	Samsung SSD 870 EVO	2TB
c	S6PNNS0T604993A	Samsung SSD 870 EVO	2TB
d	S6PNNS0T604995Z	Samsung SSD 870 EVO	2TB

r0.iad.rg.net FRR

Service Tag: 6TKCMN2
Dell R630
2x 10-Core 2.4 GHz Intel E5-2640V4
64GB DDR4 Memory
H730 Mini RAID controller
4 Gigabit RJ45 network ports
2 10Gb SFP+
iDRAC Enterprise

r0.iad Disk Drives

Unit	Model	Serial
0	Samsung SSD 870 EVO 500G	S6PWNU0Y300444D	new as of 2025.07.08
1	Samsung SSD 870 EVO 500G	S6PWNU0Y300637D	new as of 2025.07.08

r1.iad.rg.net

Dell R630 6TL9MN2 Express Service Code: 14849934302
2 x Xeon E5-2640 v4 2.4GHz 10 Cores each
64G RAM
PERC H730
BRCM 10G/GbE 2+2P 57800 - 2xCopper 2xSFP
iDRAC8

r1.iad NICs

NIC	MAC
IPMI	58:8A:5A:EE:CA:D2
Integrated NIC 1 Port 1	B8:2A:72:E0:5E:EF
Integrated NIC 1 Port 2	B8:2A:72:E0:5E:F1
Integrated NIC 1 Port 3	B8:2A:72:E0:5E:F3
Integrated NIC 1 Port 4	B8:2A:72:E0:5E:F5

r1.iad Disk Drives

for i in a b; do echo $i; smartctl -a /dev/sd$i | egrep '^(Device Model|Serial)' | cut -c 19-60; done

LUN	Serial	Model	Size
0	S7BWNJ0W650863P	Samsung SSD 870 EVO	500G
1	S7BWNJ0W685314L	Samsung SSD 870 EVO	500G

PDUs

BayTech RPC is accessed via serial from o0.iad.rg.net as pwr0/1
You will face RPC> prompt.  Type
  CONFIG
  ON x
  OFF x
  REBOOT x

pwr0.iad.rg.net (bottom PDU)

1)...r1.iad-left bad 
2)...r0.iad.rg.net   
3)...vm1.iad.rg.net  
4)...vm2.iad.rg.net  
5)...sw0.iad.rg.net  
6)...vm0.iad.rg.net

pwr1.iad.rg.net (top PDU)

1)...r1.iad-right
2)...vm2.iad.rg.net
3)...r0.iad.rg.net
4)...vm1.iad.rg.net
5)...vm0.iad.rg.net
6)...o0.iad

Decommissioned / Removed

vm1.iad.rg.net Hardware

SuperMicro AS-1022G-URF 2RU Chassis

vm1.iad.rg.net Disk Drives

From megacli -PDList -aALL | egrep 'Coerced|Inquiry'

LUN	Serial	Model	Size
0	WD-WMC130D9JJJZWDC	WD3000FYYZ-01UL1B2	2.728 TB
1	WD-WMC130D1CT76WDC	WD3000FYYZ-01UL1B2	2.728 TB
2	WD-WMC130D2MZFVWDC	WD3000FYYZ-01UL1B2	2.728 TB
3	WD-WMC130D2MZFVWDC	WD3000FYYZ-01UL1B2	2.728 TB

SprintLink

sl-mst50-ash2, ge-0/0/9 is wired to 204/4/29 with translates to Equinix panel name DC4:01:041210: CP:0204:137587 Ports 27/28. Equinix made the XC between our CFA in our cage and PSG's CFA in their cage. There in no MMR involved in the path. 2024.05.30 Jeffrey Kerrick jkerrick@cogentco.com
sl-mst50-ash2, ge-0/0/9, (To PSG peer (AS3927) CCOI: PEER PRIVATE GIGE ^psg-3927-ash2^) 2024.05.30 from BC,
port 17-18 in the Ashburn cage 1490
A-Side Patch Panel/Module/Port: PP:0103:25916||9
Z-Side Customer and Cage Number: SPRINT DC2::01490:SPRINT
Z-Side Cabinet Number: 0102
Carrier Circuit ID(if applicable): Randy Bush
Z-Side Patch Panel/Module/Port: PP:0102:605||17
Cable ID: 792604 should be used to reference that patch cable

r1.iad.rg.net Juniper MX80 from IETF DECOMISSIONED 2024.09.12

randy@r1.iad> show chassis hardware
Hardware inventory:
Item             Version  Part number  Serial number     Description
Chassis                                E4642             MX80
Midplane         REV 06   711-031594   YP5258            MX80
PEM 0            Rev 04   740-028288   UH01164           AC Power Entry Module
PEM 1            Rev 04   740-028288   WF04755           AC Power Entry Module
Routing Engine            BUILTIN      BUILTIN           Routing Engine
TFEB 0                    BUILTIN      BUILTIN           Forwarding Engine Processor
  QXM 0          REV 05   711-028408   YL0286            MPC QXM
FPC 0                     BUILTIN      BUILTIN           MPC BUILTIN
  MIC 0                   BUILTIN      BUILTIN           4x 10GE XFP
    PIC 0                 BUILTIN      BUILTIN           4x 10GE XFP
FPC 1                     BUILTIN      BUILTIN           MPC BUILTIN
  MIC 0          REV 22   750-028392   YJ7317            3D 20x 1GE(LAN) SFP
    PIC 0                 BUILTIN      BUILTIN           10x 1GE(LAN) SFP
      Xcvr 0     REV 01   740-011614   PJ717AP           SFP-LX10
      Xcvr 1     REV 01   740-031850   G1908083802       SFP-LX10
    PIC 1                 BUILTIN      BUILTIN           10x 1GE(LAN) SFP
Fan Tray                                                 Fan Tray

r0.iad.rg.net Juniper MX240 from warren Decommissioned 2025.****

randy@r0.iad> show chassis hardware
Hardware inventory:
Item             Version  Part number  Serial number     Description
Chassis                                JN11544AFAFC      MX240
Midplane         REV 07   760-021404   ABAA3443          MX240 Backplane
FPM Board        REV 03   760-021392   ABBA1738          Front Panel Display
PEM 0            Rev 01   740-022697   QCS0931C044       PS 1.2-1.7kW; 100-240V AC in
PEM 2            Rev 03   740-022697   QCS1243C071       PS 1.2-1.7kW; 100-240V AC in
Routing Engine 0 REV 07   740-015113   9009006841        RE-S-1300
CB 0             REV 05   710-021523   ABBA1993          MX SCB
FPC 2            REV 16   750-022765   XM0320            DPCE 20x 1GE + 2x 10GE R
  CPU            REV 03   710-022351   XM4341            DPC PMB
  PIC 0                   BUILTIN      BUILTIN           10x 1GE(LAN)
    Xcvr 0       REV 01   740-011613   PFF29VR           SFP-SX
    Xcvr 9       REV 02   740-011613   PGP5559           SFP-SX
  PIC 1                   BUILTIN      BUILTIN           10x 1GE(LAN)
  PIC 2                   BUILTIN      BUILTIN           1x 10GE(LAN/WAN)
  PIC 3                   BUILTIN      BUILTIN           1x 10GE(LAN/WAN)
Fan Tray 0       REV 01   710-021113   XB1539            MX240 Fan Tray

Removed

hostname	switch	oob	oobname	pwr	RU
cypress.shrubbery.net	s0.03 & s0.13	o0.35	cypress	0.4		removed 2022.08.28
akkht.hactrn.net Eth0	s0.11	o0.11	sra	0.1	2	removed 2022.08.28
akkht.hactrn.net DRAC	s0.12				2	removed 2022.08.28
r1.iad.rg.net	s0.23	o0.34	r1	equinix		Cisco 7206-G1
emu.machshav.com	s0.08	o0.40	emu	0.2		Steven M Bellovin <smb@cs.columbia.edu>
emu-2.machshav.com	s0.09/10					Steven M Bellovin <smb@cs.columbia.edu>

vm0.iad.rg.net Hardware Decommissioned

Supermicro SYS-6016T-URF with X8DTU-F mobo
Power-A&B - UL24-3#24

vm0.iad.rg.net Disk Configuration

vm0.iad.rg.net:/root# tw_cli info c0

Unit  UnitType  Status         %RCmpl  %V/I/M  Stripe  Size(GB)  Cache  AVrfy
------------------------------------------------------------------------------
u0    RAID-5    OK             -       -       256K    8381.87   RiW    ON

VPort Status         Unit Size      Type  Phy Encl-Slot    Model
------------------------------------------------------------------------------
p0    OK             u0   3.63 TB   SATA  0   -            WDC WD4003FRYZ-01F0
p1    OK             u0   3.63 TB   SATA  1   -            WDC WD4003FRYZ-01F0
p2    OK             u0   3.63 TB   SATA  2   -            WDC WD4003FRYZ-01F0
p3    OK             u0   2.73 TB   SATA  3   -            HITACHI HUA723030AL

Four drive bays seem to be numbered 0-3 left to right

From for i in 0 1 2 3; do smartctl -a -d 3ware,$i /dev/twl0 | grep Serial; done

Unit	Model	Serial
0	WD4003FRYZ-01F0DB0	VBHVEUTF	new as of 2023.05.17
1	WD4003FRYZ-01F0DB0	V1JVGGHG
2	WD4003FRYZ-01F0DB0	V1KW63UG	new as of 2023.08.09
3	HITACHI HUA723030ALA640	YVHPV0LA

2023.05.17 replaced p0 K7G9TG6L with VBHVEUTF
2022.05.18 replaced p1 WDC WD3000FYYZ-01UL serial WD-WCC130121922
2023.08.09 replaced p2 WD3000FYYZ-01UL1B0 WD-WCC130095709

2025.10.16

18 KiB Raw Blame History

Ashburn DC2 PoP

PoP Diagram

Circuits

Cogent

NTT

o0.iad

Route Views

Equipment

Rack Physical and Wiring

Ethernet Ports

RGnet Cluster / 198.180.150.4 198.180.150.10

vm0.iad.rg.net proxmox

vm1.iad.rg.net Disk Drives

vm1.iad.rg.net proxmox

vm1.iad.rg.net Disk Drives

vm2.iad.rg.net Hardware

vm2.iad Disk Drives

r0.iad.rg.net FRR

r0.iad Disk Drives

r1.iad.rg.net

r1.iad NICs

r1.iad Disk Drives

PDUs

pwr0.iad.rg.net (bottom PDU)

pwr1.iad.rg.net (top PDU)

Decommissioned / Removed

vm1.iad.rg.net Hardware

vm1.iad.rg.net Disk Drives

SprintLink

r1.iad.rg.net Juniper MX80 from IETF DECOMISSIONED 2024.09.12

r0.iad.rg.net Juniper MX240 from warren Decommissioned 2025.****

Removed

vm0.iad.rg.net Hardware Decommissioned

vm0.iad.rg.net Disk Configuration

18 KiB

Raw Blame History