Randy Bush 9c96f0ae8a fix drbd chart

2026-05-06 17:43:06 -07:00

24 KiB

Raw Blame History

Seattle Westin/NTT PoP

Block Diagram

Administrative Information

NTT Westin
2001 6th Avenue, Suite 900CA
Seattle, WA  98121
Rack D.05
Power RPP900-A2-#30 & RPP900-A1-#18 - NEMA L6-20R for NEMA L5-20P Plug

D8 is the DMARC in D. jumper to A row. would need jumper from D8 to panel in A row, and then jumper to the MMMR paenal in A row

Tripp Lite Switched PDU, 30A, 16 Outlets (5-15/20R), 120V, L5-30P, 10 ft. Cord, 2U Rack-Mount Power, TAA (PDUMH30NET)

[TrippLite PDUs](https://tripplite.eaton.com/support/PDUMH30NET
Patch Panel

Fiber Paths

NTT MMR Cross-Connects (from RGnet in NTT Rack D.05)
    SIX 10G RGnet
        C164428: 19TH FL MMR / MMR / 305 / 1901.03.05.05 15,16 SC
        to 900 / A (NTT) / A.04 / PANEL 9 PORT B2
    SIX 1G PEERING
        C164427: 19TH FL MMR / MMR / 102 / 1901.01.02.03 23,24 SC
        to 900 / A (NTT) / A.03 / PANEL 1 PORT B6
    Cogent 1G RGnet (Cogent Order Number 3-002222510)
        C164443: 19TH FL MMR / MMR / 903 / 1901.09.03.01 13,14 SC
        to 900 / A (NTT) / A.04 / PANEL 12 PORT B1

XC from RGnet in D.05 to NTT 9th floor DMARC in D.08
XC from NTT D.08 to 19th floor NTT/MMR Panel in A.03/A.04
XC from MMR on 19th to Cogent and SIX

MMR XCs:
    GIN CID: C164427
    DRT CID: 01157542
    NTT TDI: 1901.01.02.03 Port #23/24
    SIX TDI: 1901.06.02.01 Ports 23/24 (B6)

    GIN CID: C164428
    DRT CID: 01157543
    NTT TDI: 1901.03.05.05 Port #15/16
    SIX TDI: 1901.06.02.01 Ports 33/34 (C5)

D.05 Patch Panel Port Assignments
  5-6  SIX 10G C164428
  7-8  SIX  1G C164427  to sw3-46
  9-10 NTT
 11-12 Cogent

Rack and Stack

RU	Name	Device	Power	IPMI/iDRAC	1st Port	2nd Port	Serial
40	Fiber Patch
39
38	r2.sea.rg.net	Arrcus S9600-72XC-R	0.09 & 1.09	ma1 sw4.36	swp.6 patch 5-6	swp.20 sw4.47	o0.9
37	2RU	UfiSpace
36	sw4.sea.rg.net	Cisco 3064 Switch	0.3 & 1.3	sw0.22 ???	sw4.48-sw3.47		o0.8
35
34	r1.sea.rg.net	Dell 630 / FRR	0.10 & 1.10	????	sw4.17	patch 11-12	o0.10
33	r0.sea.rg.net	Cisco ASR1001	0.01 & 1.01		sw4.35	patch 9-10	o0.11
32
31	seattle01.mux.peering.usc.edu	Supermicro	0.08	sw0.01	sw0.11	sw0.38	-
30	2RU					patch 7-8-sw3.40
29	pwr1.sea.rg.net	TrippLite PDUMH30NET	A1.-#18	sw0.14	sw0.16		-
28	2RU
27
26	pwr0.sea.rg.net	TrippLite PDUMH30NET	A2-#30	sw0.12	sw0.12		????
25	2RU
24
23	sw3.sea.rg.net	Cisco 3064 Switch	0.02&1.02	sw0.21	sw3.47->sw4.48	sw3.48-sw0.47	o0.7
22
21	sw0.sea.rg.net	Cisco WS-C3750G-POE-48	1.08	????	sw0.47→sw3.48		o0.6
20
19	vm5.sea.rg.net	Dell 640	0.15 & 1.15	sw0.02	sw3.43	sw3.44	o0.13
18	vm4.sea.rg.net	Dell 630	0.14 & 1.14	sw0.05	sw3.01	sw3.02	o0.13
17	vm3.sea.rg.net	SuperMicro 6019U-TRT	0.13 & 1.13	sw0.19	sw3.20	sw3.22	????
16
15	vm2.sea.rg.net	Supermicro AS-2022G-URF	0.12 & 1.12	sw0.08	sw0.41	sw3.31	o0.5
14	2RU
13	vm1.sea.rg.net	being replaced				sw3.30	o0.4
12	vm0.sea.rg.net	SuperMicro w/ X7DWU	1.16	sw0.39	sw0.26	sw3.29 down	o0.3
11
10	o0.sea.rg.net	Cyclades ACS-48	0.16	sw0.44	sw0.43
9
8	raid0.sea.rg.net	SuperMicro 5018R-WR	0.11 & 1.11	sw0.09	sw0.10		o0.12
7	ap.sea.rg.net	tiny blue thing, not in RU-7	peering USB		sw0.3
6
5
4
3
2
1

vm[0-5].sea.rg.net Ganeti Cluster

gnt-instance list -o "name,pnode,snodes,sda_size,oper_ram,vcpus"

vm0.sea.rg.net:/root# ./do-stat
Node           DTotal DFree MTotal MNode  MFree Pinst Sinst
vm0.sea.rg.net  10.9T  4.7T 125.8G 88.0G 101.6G     8     5
vm2.sea.rg.net   5.5T  4.6T  94.4G 13.1G  86.6G     4     5
vm3.sea.rg.net  10.9T  7.1T 503.8G 18.3G 484.2G     3     0
vm4.sea.rg.net   3.6T  2.2T 251.8G 29.8G 240.5G     6     4
vm5.sea.rg.net   7.3T  1.1T 188.5G 75.9G 152.3G     9     8

Instance                     Primary_node   Secondary_Nodes  Disk/0 MemoryVCPUs
a1000004.rg.net              vm5.sea.rg.net                    8.0G   2.0G    1
adrilankha.hactrn.net        vm5.sea.rg.net vm0.sea.rg.net   260.0G   4.0G    1
archive.psg.com              vm0.sea.rg.net vm5.sea.rg.net   128.0G   2.0G    1
build-u.rpki.net             vm5.sea.rg.net vm2.sea.rg.net   100.0G   1.0G    1
chezrandy.x0.dk              vm2.sea.rg.net                  300.0G   2.0G    1
hans.rg.net                  vm4.sea.rg.net vm0.sea.rg.net   251.0G   4.0G    2
hiroshima.bogus.com          vm5.sea.rg.net vm0.sea.rg.net   250.0G   2.0G    1
lucky.bgp.nu                 vm5.sea.rg.net vm2.sea.rg.net    20.0G   4.0G    1
lxd0.sea.rg.net              vm4.sea.rg.net                   32.0G      -    8
m1.rg.net                    vm4.sea.rg.net                   32.0G   3.0G    1
mon1.rg.net                  vm0.sea.rg.net vm5.sea.rg.net   100.0G   4.0G    1
nagasaki.bogus.com           vm0.sea.rg.net vm5.sea.rg.net   250.0G   4.0G    1
netsrv.sea.rg.net            vm4.sea.rg.net vm5.sea.rg.net   210.0G   1.0G    1
nlring.sea.rg.net            vm2.sea.rg.net vm5.sea.rg.net    32.0G   2.0G    1
ns0.rg.net                   vm3.sea.rg.net vm4.sea.rg.net    32.0G   8.0G    1
ns1.securerouting.org        vm2.sea.rg.net                  100.0G   2.0G    1
okui.psg.com                 vm0.sea.rg.net vm5.sea.rg.net  1001.0G   1.0G    1
pfs.sea.rg.net               vm3.sea.rg.net                    8.0G   1.0G    1
psg.com                      vm5.sea.rg.net vm0.sea.rg.net     2.0T  16.0G    3
pws1                         vm0.sea.rg.net vm5.sea.rg.net   500.0G   8.0G    1
rcuss-build.rg.net           vm3.sea.rg.net                  316.0G  64.0G   16
rip.psg.com                  vm5.sea.rg.net vm4.sea.rg.net    32.0G  32.0G    8
shout.rg.net                 vm5.sea.rg.net vm2.sea.rg.net    24.0G   2.0G    1
tamias.rg.net                vm0.sea.rg.net vm2.sea.rg.net    17.0G      -    1
turing.worldpowersystems.com vm2.sea.rg.net vm4.sea.rg.net   250.0G   2.0G    1
vault.sea.rg.net             vm4.sea.rg.net vm2.sea.rg.net     8.0G   1.0G    1
viz2.rg.net                  vm0.sea.rg.net vm4.sea.rg.net    64.0G   4.0G    1
wg.sea.rg.net                vm5.sea.rg.net                   10.0G   2.0G    1
zoe.dns.gh                   vm4.sea.rg.net vm0.sea.rg.net    32.0G   1.0G    1
zzyzx.sigpipe.org            vm0.sea.rg.net vm5.sea.rg.net   100.0G   2.0G    1

Guest VMs on vm0/2/3/4/5.sea.rg.net / 147.28.0.14/15/16/29

Instance	IP_Address	SysAdmin
a1000004.rg.net	147.28.0.33	Randy Bush randy@psg.com
adrilankha.hactrn.net	147.28.0.19	Rob Austein sra@hactrn.net
archive.psg.com	147.28.0.8	Randy Bush randy@psg.com
build-u.rpki.net	147.28.0.28	Rob Austein sra@hactrn.net
chezrandy.x0.dk	147.28.0.56	Phil Regnauld pr@x0.dk
hans.rg.net	147.28.0.42	Hans Kuhn hak@flyingpoodle.com
hiroshima.bogus.com	147.28.0.82	Hans Kuhn <hak@flyingpoodle.com
lucky.bgp.nu	147.28.0.57	Paul Southworth pwsouth@gmail.com
lxd0.sea.rg.net		Hans Kuhn <hak@flyingpoodle.com
m1.rg.net	147.28.0.30	Randy Bush randy@psg.com
nagasaki.bogus.com	147.28.0.81	Joel Jaeggli joelja@bogus.com
netsrv.sea.rg.net	147.28.0.7	Randy Bush randy@psg.com
nlring.sea.rg.net	147.28.0.89	Randy Bush randy@psg.com
ns0.rg.net	147.28.1.39	Randy Bush randy@psg.com
ns1.securerouting.org	147.28.0.45	Sam Weiler weiler@watson.org
okui.psg.com	147.28.0.25	Patrick Okui pokui@psg.com
pfs.sea.rg.net	147.28.0.32	Randy Bush randy@psg.com
psg.com	147.28.0.62	Randy Bush randy@psg.com
pws1	147.28.0.53	Paul Southworth pwsouth@gmail.com
rcuss-build.rg.net	147.28.0.170	Rob Austein sra@hactrn.net
rip.psg.com	147.28.0.39	Randy Bush randy@psg.com
shout.rg.net	147.28.0.38	Randy Bush randy@psg.com
tamias.rg.net	147.28.0.80	Randy Bush randy@psg.com
turing.worldpowersystems.com	147.28.0.17	Tom Jennings tom@sr-ix.com
vault.sea.rg.net	147.28.0.2	Randy Bush randy@psg.com
viz2.rg.net	147.28.0.27	Stephen North scnorth@gmail.com
wg.sea.rg.net	147.28.0.11	Randy Bush randy@psg.com
zoe.dns.gh	147.28.0.23	Joel Jaeggli joelja@bogus.com
zzyzx.sigpipe.org	147.28.0.13	Michael Elkins melkins@sigpipe.org

Back/DRBD Channel

net 10, drdb Ports for vm[0-4]

Switch	Port	Host	Ether	IP Address
sw0	39	vm0.sea	eth1	10.0.0.100
sw0	--	vm1.sea	eth5	10.0.0.101
sw0	08	vm2.sea	eth1	10.0.0.102
sw0	19	vm3.sea	eth1	10.0.0.103
sw0	05	vm4.sea	enp130s0f1	10.0.0.104
sw0	02	vm5.sea	enp216s0f1	10.0.0.105

vm0.sea.rg.net Hardware

SuperMicro SuperServer 6018R-MT Chassis
SuperMicro X10DRL-i MotherBoard
2 Intel Xeon E5-2630 V4 2.2 GHz LGA 2011 85W
128G Kingston ValueRAM 64GB (4 x 16GB) 288-Pin DDR4 SDRAM ECC Registered DDR4 2133 (PC4 17000) Server Memory Model KVR21R15D4K4/64

vm0.sea drives, numbered left to right

From for i in a b c d; do smartctl -a /dev/sd$i | egrep 'Model|Serial|Capacity'; done

Bay	Model	Serial
0	WD6002FFWX-68TZ4N0	NCGW1TVV	2016.09.24
1	WD6002FFWX-68TZ4N0	NCGWP5SV	2016.09.24
2	WD6002FFWX-68TZ4N0	NCGWD6NV	2016.09.24
3	HGST HUS726T6TALE6L4	V7GESAHH	2024.03.01

vm2.sea.rg.net Hardware

SuperMicro AS-2022G-URF 2RU Chassis

vm2.sea.rg.net Disk Drives

vm2.sea.rg.net:/home/randy# megacli -PDList -aALL | egrep '(Slot|^Device Id:|Inquiry)'
Slot Number: 0
Device Id: 11
Inquiry Data:      WD-WCC6N3FVKC2SWDC WD2002FFSX-68PF8N0                  81.00A81
Slot Number: 1
Device Id: 10
Inquiry Data:       MK0251YGKB56HAHitachi HUA723020ALA640                 MK7OAA10
Slot Number: 2
Device Id: 9
Inquiry Data:       MK0251YGKB4N2AHitachi HUA723020ALA640                 MK7OAA10
Slot Number: 3
Device Id: 8
Inquiry Data:       MK0251YGKB58LAHitachi HUA723020ALA640                 MK7OAA10
Slot Number: 4
Device Id: 12
Inquiry Data:       MJ1311YNG6XEVAHitachi HDS5C3030ALA630                 MEAOA5C0


vm2.sea.rg.net:/home/randy# smartctl -a -d sat+megaraid,11  /dev/sda
smartctl 6.6 2016-05-31 r4324 [x86_64-linux-4.9.0-13-amd64] (local build)
Copyright (C) 2002-16, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF INFORMATION SECTION ===
Device Model:     WDC WD2002FFSX-68PF8N0
Serial Number:    WD-WCC6N3FVKC2S
LU WWN Device Id: 5 0014ee 2687b471f
Firmware Version: 81.00A81
User Capacity:    2,000,398,934,016 bytes [2.00 TB]
Sector Sizes:     512 bytes logical, 4096 bytes physical
Rotation Rate:    7200 rpm
Form Factor:      3.5 inches
Device is:        Not in smartctl database [for details use: -P showall]
ATA Version is:   ACS-3 T13/2161-D revision 5
SATA Version is:  SATA 3.1, 6.0 Gb/s (current: 6.0 Gb/s)
Local Time is:    Thu Aug 19 20:46:26 2021 UTC
SMART support is: Available - device has SMART capability.
SMART support is: Enabled

LUN	Serial	Model	Size
0	WCC6N3FVKC2S	WD2002FFSX-68PF8N0	1.818 TB
1	MK0251YGKB56HA	MK7OAA10	1.818 TB
2	MK0251YGKB4N2A	MK7OAA10	1.818 TB
3	MK0251YGKB58LA	MK7OAA10	1.818 TB
4	MJ1311YNG6XEVA	MEAOA5C0	2.728 TB

2021.08.19 LUN0 drive MK0171YFK3TVGA removed

vm3.sea.rg.net Hardware

SuperMicro SuperServer 6019U-TRT

X10DRW-iT Motherboard
Two Intel Ten Core 2.4Ghz Xeon 25MB cache 8.0GT/s QPI (90W) (E5-2640v4)
128G RAM - 8x16GB DDR4 2400Mhz 19200 ECC/REG
Four HGST 6TB Enterprise SATA3 6Gb/s 7200RPM 128MB

vm3.sea drives, we guess numbered left to right

From for i in a b c d; do smartctl -a /dev/sd$i | egrep 'Model|Serial|Capacity'; done

Bay	Model	Serial	Capacity	Installed
0	HGST HUS726060ALE610	K1J48AGD	6TB	2017.08.17
1	HGST HUS726060ALE610	K1J2PTND	6TB	2017.08.17
2	HGST HUS726060ALE610	K8J6HV2N	6TB	2022.09.15
3	HGST HUS726060ALE610	K1J46RHD	6TB	2017.08.17

replaced: | 2 | HGST HUS726060ALA640 | AR11001EV17LLB | 6TB | 2022.09.15 |

for i in b c d e; do echo $i; smartctl -a /dev/sd$i | egrep '(^Serial|Extended)'; done

vm4.sea.rg.net Hardware

Dell Poweredge R630 Server 2xE5-2640V4 10-core 2.4GHz, 256GB RAM, H730
Service tag: 6TL7MN2
PERC H730 disk controller
Express Service Code 14849840990
How to configure the DELL PERC H700/H710/H730/H730P RAID controller
Findiing a DIMM

Bay	Model	Serial	Capacity	Installed
a	Samsung SSD 870 EVO 2TB	S6PNNL0T909370N	2TB	22.11.18
b	Samsung SSD 870 EVO 2TB	S6PNNL0T909374Z	2TB	22.11.18
c	Samsung SSD 870 EVO 2TB	S6PNNL0T708760T	2TB	22.11.18
d	Samsung SSD 870 EVO 2TB	S6PNNL0T906382W	2TB	22.11.18

vm5.sea.rg.net (vm1 Replacement)

Dell Poweredge R640 server
Service Tag: CQPCHQ2
2x 10-Core 2.20 GHz Intel CPU's Installed (Silver 4114)
192GB DDR4 (6x32GB) memory installed
PERC H730P Mini RAID controller.
4 x Samsung 4TB EVO 870 SSDs
2x power supplies, 750W each, 100-240V
4x Gigabit RJ45 NIC network ports

vm5 Disks

for i in a b c d; do echo $i; smartctl -a /dev/sd$i | egrep '(^Serial|Model)'; done

Bay	Model	Serial	Capacity	Installed
a	Samsung SSD 870 EVO 4TB	S6PJNS0W307070P	4TB	23.05.21
b	Samsung SSD 870 EVO 4TB	S6PJNS0W307067B	4TB	23.05.21
c	Samsung SSD 870 EVO 4TB	S6PJNS0W307069T	4TB	23.05.21
d	Samsung SSD 870 EVO 4TB	S6PJNS0W307064R	4TB	23.05.21

raid0.sea.rg.net

5018R-WR Chassis + MoBo
X10SRW-F Motherboard
Intel Xeon® Processor E5-2609 v4
Samsung DDR4 2133MHzCL15 16GB RegECC 2Rx4 (PC4 2133) Internal Memory M393A2G40DB0-CPB
FreeBSD USB key left in USB port

r1.sea.rg.net Dell/FRR

Dell R630 GM12D42 Express Service Code: 36160563170
2x 10-Core 2.4 GHz Intel CPU's Installed (E5-2640V4)
64G RAM 4x16G
PERC H730 Mini RAID controller
4x 1 Gigabit RJ45 NIC
Intel X520-SR2 2-Port 10G SFP+ Ethernet, PCI-e
iDRAC8

r1.sea Haed Drives

Bay	Model	Serial	Capacity	Installed
a	Samsung SSD 870 EVO 512G	S6PXNS0L100210	512G	2026.02.27
b	Samsung SSD 870 EVO 512G	S6PXNS0L101546	512G	2026.02.27

raid0.sea.rg.net Drive Bays

From for i in {0..3}; do smartctl -a /dev/ada$i | egrep 'Model|Serial|Capacity'; done

LUN	Model	Serial	Capacity	Installed
0	WDC WD60EFRX-68L0BN1	WD-WX41DA5F88N9	6tb	2018.12.09
1	WDC WD6004FRYZ-01CBJB0	AQ01LV7M	6tb	2024.09.26
2	WDC WD60EFRX-68L0BN1	WD-WX11D3678XTL	6tb	2018.12.09
3	HGST HUS726T6TALE6L4	V70PUZ0K	6tb	2025.12.01

Removed

LUN	Model	Serial	Capacity	Installed	Replaced
1	WDC WD60EFRX-68L0BN1	WD-WX11D363T2Y5	6tb	2018.12.09	2024.07.10
1	WDC WD6004FRYZ-01CBJB0	AQ00WHDM	6tb	2024.07.10	2024.09.26
3	WDC WD60EFRX-68L0BN1	WD-WX41D75LN9KC	6tb	2018.12.09	died 2025.11.28

Switches

name	mfr	model	ip sddress
sw0.sea.rg.net	Cisco	WS-C3750G-48PS	147.28.0.253
sw3.sea.rg.net	Cisco	Nexus3064 48x10GE +4	147.28.0.239
sw4.sea.rg.net	Cisco	Nexus3064 48x10GE +4	147.28.0.238

Removed

sw1.sea.rg.net	SMC	SMC8024L2	147.28.0.252
sw2.sea.rg.net	Cisco	WS-C3750G-24PS	147.28.0.246

Spare Drives


spares in westin

HGST	H3D30003254S	YNG1E3ZC	3TB
Seagate	ST31000528AS	6VPCVN1Y	1TB
Seagate	ST3750640A8	3QD06D5B	750G
HGST	118032816	YFJLBZ5A	2TB

r1.sea gigether pic left to right

port	to	medium
ge-0/0/0	LAN	copper
ge-0/0/1	six	fiber sfp
	unused	copper sfp
	unused

2026.05.06

24 KiB Raw Blame History Unescape Escape

Seattle Westin/NTT PoP

Block Diagram

Administrative Information

Fiber Paths

Rack and Stack

vm[0-5].sea.rg.net Ganeti Cluster

Guest VMs on vm0/2/3/4/5.sea.rg.net / 147.28.0.14/15/16/29

Back/DRBD Channel

vm0.sea.rg.net Hardware

vm0.sea drives, numbered left to right

vm2.sea.rg.net Hardware

vm2.sea.rg.net Disk Drives

vm3.sea.rg.net Hardware

vm3.sea drives, we guess numbered left to right

vm4.sea.rg.net Hardware

vm5.sea.rg.net (vm1 Replacement)

vm5 Disks

raid0.sea.rg.net

r1.sea.rg.net Dell/FRR

r1.sea Haed Drives

raid0.sea.rg.net Drive Bays

Removed

Switches

Removed

Spare Drives

r1.sea gigether pic left to right

24 KiB

Raw Blame History