Randy Bush dbb0d8f543 psg.com virtualized

2024-11-29 11:22:10 -08:00

21 KiB

Raw Blame History

Seattle KOMO PoP

Block Diagram

Administrative Information

Evocative at KOMO Plaza
140 4th Ave North, Suite 200
Seattle WA 98109
support@evocative.com
+1 888 365-2656 (opt 3 is NOC)
+1 206 290-9206 (on-site tech)

Rack SEF.A.0F.012

Fiber Patch

SIX 10G to r2,sea.rg.net
  A.OF.12.01
  OCSX2.A05
  MR5A.04.A05
  MR5D-09.M06

Cogent 1G to r1.sea.rg.net
  Gi100/0/0/2-rcr61.b007274-0.sea02
  MR5D.09.C06 (Cogent port on patch panel in meet-me-room)
  MR5A.04.A06 (eviocative port on patch panel in meet-me-room)
  OCXS2.A06
  A.OF.12.02 (patch panel port in RGnet rack)
  
  Cogent Customer Support: 1-877-726-4368, support@cogentco.com        

SIX 1G to peering.usc.edu
  A.OF.12.03
  OCSX1.M06
  MR5A.02.M07
  MR5d.09.D06

KOMO Rack

Rack SEF.A.0F.012 Physical Hosts and Wiring

hostname	switch	ipmi	oob	oobname	pwr	rack	Model
sw4.sea.rg.net	s3.47		0.08	sw4	3L 3R	41	Cisco Nexus-3064
rcuss0.rg.net	s0.23	N/A	0.01	rcuss0	4L 4R	40	Redstone
vm3.sea.rg.net	s4.20/s4.22	s4.18	????	vm3	5L 5R	39	SuperMicro SuperServer 6019U-TRT
r2.sea.rg.net	s4.47 s4.36		0.09	r2	6L 6R	37-38	Arrcus S9600-72XC-R q2c
vm4.sea.rg.net	s4.45/s4.46	s3.5	0.13	NOOP	9L 9R	36	Dell 630
vm2.sea.rg.net	s0.41/s0.31	s0.41	0.05	vm2	10L 10R	34-35	SuperMicro AS-2022G-URF 2RU Chassis
psg.com	s0.13	s0.19	0.02	psg	7L	33	(see below)
seattle01.mux.peering.usc.edu	s0.11 s0.38	0.20			11R	31	Supermicro Ethan Katz-Bassett <ethanbkb@gmail.com
vm5.sea.rg.net	s4.45/s4.46	s0.8	0.13	vm5	19L 19R	30	Dell 640
sw3.sea.rg.net	s0.47		0.07	sw3	12L 12R	27	Cisco Nexus-3064
sw0.sea.rg.net	s3.48		0.06	sw0	13L	26	Cisco WS-C3750G-POE-48
atlas0.sea.rg.net	s0.25					20.5	Atlas v0 Probe
raid0.sea.rg.net	s0.27	s0.03	0.12	raid0	17L 17R	19	SuperMicro 5018R-WR
o0.sea.rg.net	s0.06				13R	18	Cyclades ACS-48
vm0.sea.rg.net	s0.26/s0.29	s0.39	0.03	vm0	18R	15	Supermicro - w/ X7DWU
r0.sea.rg.net	s0.35/46		0.11	r0	23L 23R	7	Cisco ASR1001 commissioned 2018.02.02
r1.sea.rg.net	s0.17		0.10	r1	24L 24R	5-6	Juniper M7i
pwr0.sea.rg.net	s0.12					L	APC AP7930
pwr1.sea.rg.net	s0.16					R	APC AP7930

Decommissioned:

| vm1.sea.rg.net | s0.34/s3.30 | s0.33 | 0.04 | NOOP | 18L | 16-17 | (see below) |

pwr0 and pwr1 MACs

name	side	model	MAC
pwr0	left	AP7930	00:C0:B7:C8:49:BC
pwr0	right	AP7930	00:C0:B7:30:1C:16

100mb interfaces on sw0

atlas probe
o0.sea.rg.net
four power bars
psg.com ipmi
peering.usc.edu ipmi
ipmi.vm1.sea.rg.net
ipmi.vm0.sea.rg.net
pwr0.sea.rg.net
pwr1.sea.rg.net

vm[0-5].sea.rg.net Ganeti Cluster

gnt-instance list -o "name,pnode,snodes,sda_size,oper_ram,vcpus"

vm0.sea.rg.net:/root# ./do-stat
Node           DTotal DFree MTotal  MNode  MFree Pinst Sinst
vm0.sea.rg.net  10.9T  5.6T 125.8G  89.8G 102.3G     8     5
vm2.sea.rg.net   5.5T  4.4T  94.4G  21.2G  83.1G     5     7
vm3.sea.rg.net  10.9T  6.6T 503.8G 427.0G 431.6G     4     2
vm4.sea.rg.net   3.6T  2.7T 251.8G  58.1G 241.1G     4     3
vm5.sea.rg.net   7.3T  1.9T 188.5G  91.2G 159.9G    10     8

Instance              Primary_node   Secondary_Nodes  Disk/0 Memory CPUs
a1000004.rg.net       vm5.sea.rg.net                    8.0G   2.0G    1
adrilankha.hactrn.net vm5.sea.rg.net vm0.sea.rg.net   260.0G   4.0G    1
archive.psg.com       vm5.sea.rg.net vm3.sea.rg.net   200.0G   2.0G    1
build-u.rpki.net      vm5.sea.rg.net vm2.sea.rg.net   100.0G   1.0G    1
chezrandy.x0.dk       vm2.sea.rg.net                  300.0G   2.0G    1
gogs.rg.net           vm3.sea.rg.net vm2.sea.rg.net    65.0G   2.0G    1
hans.rg.net           vm4.sea.rg.net vm0.sea.rg.net   251.0G   4.0G    2
hiroshima.bogus.com   vm5.sea.rg.net vm0.sea.rg.net   250.0G   2.0G    1
lucky.bgp.nu          vm5.sea.rg.net vm2.sea.rg.net    20.0G   4.0G    1
mon1.rg.net           vm0.sea.rg.net vm5.sea.rg.net   100.0G   4.0G    1
nagasaki.bogus.com    vm0.sea.rg.net vm5.sea.rg.net   250.0G   4.0G    1
netsrv.sea.rg.net     vm4.sea.rg.net vm5.sea.rg.net   210.0G   1.0G    1
nlring.sea.rg.net     vm2.sea.rg.net vm5.sea.rg.net    32.0G   2.0G    1
ns0.rg.net            vm3.sea.rg.net vm4.sea.rg.net    32.0G   8.0G    1
ns1.securerouting.org vm2.sea.rg.net                  100.0G   2.0G    1
okui.psg.com          vm0.sea.rg.net vm5.sea.rg.net  1001.0G   1.0G    1
pfs.sea.rg.net        vm3.sea.rg.net                    8.0G   1.0G    1
psg.com               vm5.sea.rg.net vm0.sea.rg.net     2.0T  16.0G    1
pws0                  vm5.sea.rg.net vm2.sea.rg.net   100.0G   4.0G    1
pws1                  vm0.sea.rg.net vm5.sea.rg.net   500.0G   8.0G    1
rcuss-build.rg.net    vm3.sea.rg.net                  316.0G  64.0G   16
rip1.psg.com          vm2.sea.rg.net vm5.sea.rg.net    36.0G   4.0G    1
rip.psg.com           vm5.sea.rg.net vm4.sea.rg.net    32.0G  32.0G    8
shout.rg.net          vm5.sea.rg.net vm2.sea.rg.net    16.0G   1.0G    1
tamias.rg.net         vm0.sea.rg.net vm2.sea.rg.net    17.0G   2.0G    1
turing.sr-ix.com      vm2.sea.rg.net vm3.sea.rg.net   256.0G   2.0G    1
vault.sea.rg.net      vm4.sea.rg.net vm2.sea.rg.net     8.0G   1.0G    1
viz2.rg.net           vm0.sea.rg.net vm4.sea.rg.net    64.0G   4.0G    1
work0.psg.com         vm0.sea.rg.net                  200.0G   4.0G    1
zoe.dns.gh            vm4.sea.rg.net vm0.sea.rg.net    32.0G   1.0G    1
zzyzx.sigpipe.org     vm0.sea.rg.net vm5.sea.rg.net   100.0G   2.0G    1

Guest VMs on vm0/2/3/4/5.sea.rg.net / 147.28.0.14/15/16/29

Instance	IP_Address	SysAdmin
a1000004.rg.net	147.28.0.33	Randy Bush randy@psg.com
adrilankha.hactrn.net	147.28.0.19	Rob Austein sra@hactrn.net
archive.psg.com	147.28.0.8	Randy Bush randy@psg.com
bbgp.rg.net	147.28.0.60	Randy Bush randy@psg.com
build-u.rpki.net	147.28.0.28	Rob Austein sra@hactrn.net
chezrandy.x0.dk	147.28.0.56	Phil Regnauld pr@x0.dk
gogs.rg.net	147.28.0.10	Randy Bush randy@psg.com
hans.rg.net	147.28.0.42	Hans Kuhn hak@flyingpoodle.com
hiroshima.bogus.com	147.28.0.82	Hans Kuhn <hak@flyingpoodle.com
lucky.bgp.nu	147.28.0.57	Paul Southworth pwsouth@gmail.com
mon1.rg.net	147.28.0.35	Randy Bush randy@psg.com
nagasaki.bogus.com	147.28.0.81	Joel Jaeggli joelja@bogus.com
netsrv.sea.rg.net	147.28.0.7	Randy Bush randy@psg.com
nlring.sea.rg.net	147.28.0.89	Randy Bush randy@psg.com
okui.psg.com	147.28.0.25	Patrick Okui pokui@psg.com
pfs.sea.rg.net	147.28.0.32	Randy Bush randy@psg.com
psg.com	147.28.0.62	Randy Bush randy@psg.com
pws0	147.28.0.52	Paul Southworth pwsouth@gmail.com
pws1	147.28.0.53	Paul Southworth pwsouth@gmail.com
r1.securerouting.org	147.28.0.131	Sam Weiler weiler@watson.org
rcuss-build.rg.net	147.28.0.170	Rob Austein sra@hactrn.net
rip.psg.com	147.28.0.39	Randy Bush randy@psg.com
rip1.psg.com	147.28.0.31	Randy Bush randy@psg.com
shout.rg.net	147.28.0.38	Randy Bush randy@psg.com
tamias.rg.net	147.28.0.80	Randy Bush randy@psg.com
turing.sr-ix.com	147.28.0.17	Tom Jennings tom@sr-ix.com
vault.sea.rg.net	147.28.0.2	Randy Bush randy@psg.com
viz2.rg.net	147.28.0.27	Stephen North scnorth@gmail.com
work0.psg.com	147.28.0.11	Randy Bush randy@psg.com
xmpp.rg.net	147.28.0.6	Randy Bush randy@psg.com
zoe.dns.gh	147.28.0.23	Joel Jaeggli joelja@bogus.com
zzyzx.sigpipe.org	147.28.0.13	Michael Elkins melkins@sigpipe.org

Back/DRBD Channel

net 10, drdb Ports for vm[0-4]

Switch	Port	Host	Ether	IP Address
sw0	29	vm0.sea	eth1	10.0.0.100
sw0	30	vm1.sea	eth5	10.0.0.101
sw0	31	vm2.sea	eth1	10.0.0.102
sw4	22	vm3.sea	eth1	10.0.0.103
sw4	46	vm4.sea	enp130s0f1	10.0.0.104
sw4	45	vm5.sea	enp216s0f1	10.0.0.105

vm0.sea.rg.net Hardware

SuperMicro SuperServer 6018R-MT Chassis
SuperMicro X10DRL-i MotherBoard
2 Intel Xeon E5-2630 V4 2.2 GHz LGA 2011 85W
128G Kingston ValueRAM 64GB (4 x 16GB) 288-Pin DDR4 SDRAM ECC Registered DDR4 2133 (PC4 17000) Server Memory Model KVR21R15D4K4/64

vm0.sea drives, numbered left to right

From for i in a b c d; do smartctl -a /dev/sd$i | egrep 'Model|Serial|Capacity'; done

Bay	Model	Serial
0	WD6002FFWX-68TZ4N0	NCGW1TVV	2016.09.24
1	WD6002FFWX-68TZ4N0	NCGWP5SV	2016.09.24
2	WD6002FFWX-68TZ4N0	NCGWD6NV	2016.09.24
3	HGST HUS726T6TALE6L4	V7GESAHH	2024.03.01

vm2.sea.rg.net Hardware

SuperMicro AS-2022G-URF 2RU Chassis

vm2.sea.rg.net Disk Drives

vm2.sea.rg.net:/home/randy# megacli -PDList -aALL | egrep '(Slot|^Device Id:|Inquiry)'
Slot Number: 0
Device Id: 11
Inquiry Data:      WD-WCC6N3FVKC2SWDC WD2002FFSX-68PF8N0                  81.00A81
Slot Number: 1
Device Id: 10
Inquiry Data:       MK0251YGKB56HAHitachi HUA723020ALA640                 MK7OAA10
Slot Number: 2
Device Id: 9
Inquiry Data:       MK0251YGKB4N2AHitachi HUA723020ALA640                 MK7OAA10
Slot Number: 3
Device Id: 8
Inquiry Data:       MK0251YGKB58LAHitachi HUA723020ALA640                 MK7OAA10
Slot Number: 4
Device Id: 12
Inquiry Data:       MJ1311YNG6XEVAHitachi HDS5C3030ALA630                 MEAOA5C0


vm2.sea.rg.net:/home/randy# smartctl -a -d sat+megaraid,11  /dev/sda
smartctl 6.6 2016-05-31 r4324 [x86_64-linux-4.9.0-13-amd64] (local build)
Copyright (C) 2002-16, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF INFORMATION SECTION ===
Device Model:     WDC WD2002FFSX-68PF8N0
Serial Number:    WD-WCC6N3FVKC2S
LU WWN Device Id: 5 0014ee 2687b471f
Firmware Version: 81.00A81
User Capacity:    2,000,398,934,016 bytes [2.00 TB]
Sector Sizes:     512 bytes logical, 4096 bytes physical
Rotation Rate:    7200 rpm
Form Factor:      3.5 inches
Device is:        Not in smartctl database [for details use: -P showall]
ATA Version is:   ACS-3 T13/2161-D revision 5
SATA Version is:  SATA 3.1, 6.0 Gb/s (current: 6.0 Gb/s)
Local Time is:    Thu Aug 19 20:46:26 2021 UTC
SMART support is: Available - device has SMART capability.
SMART support is: Enabled

LUN	Serial	Model	Size
0	WCC6N3FVKC2S	WD2002FFSX-68PF8N0	1.818 TB
1	MK0251YGKB56HA	MK7OAA10	1.818 TB
2	MK0251YGKB4N2A	MK7OAA10	1.818 TB
3	MK0251YGKB58LA	MK7OAA10	1.818 TB
4	MJ1311YNG6XEVA	MEAOA5C0	2.728 TB

2021.08.19 LUN0 drive MK0171YFK3TVGA removed

vm3.sea.rg.net Hardware

SuperMicro SuperServer 6019U-TRT

X10DRW-iT Motherboard
Two Intel Ten Core 2.4Ghz Xeon 25MB cache 8.0GT/s QPI (90W) (E5-2640v4)
128G RAM - 8x16GB DDR4 2400Mhz 19200 ECC/REG
Four HGST 6TB Enterprise SATA3 6Gb/s 7200RPM 128MB

vm3.sea drives, we guess numbered left to right

From for i in a b c d; do smartctl -a /dev/sd$i | egrep 'Model|Serial|Capacity'; done

Bay	Model	Serial	Capacity	Installed
0	HGST HUS726060ALE610	K1J48AGD	6TB	2017.08.17
1	HGST HUS726060ALE610	K1J2PTND	6TB	2017.08.17
2	HGST HUS726060ALE610	K8J6HV2N	6TB	2022.09.15
3	HGST HUS726060ALE610	K1J46RHD	6TB	2017.08.17

replaced: | 2 | HGST HUS726060ALA640 | AR11001EV17LLB | 6TB | 2022.09.15 |

for i in b c d e; do echo $i; smartctl -a /dev/sd$i | egrep '(^Serial|Extended)'; done

vm4.sea.rg.net Hardware

Dell Poweredge R630 Server 2xE5-2640V4 10-core 2.4GHz, 256GB RAM, H730
Service tag: 6TL7MN2
PERC H730 disk controller
Express Service Code 14849840990
How to configure the DELL PERC H700/H710/H730/H730P RAID controller
Findiing a DIMM

Bay	Model	Serial	Capacity	Installed
a	Samsung SSD 870 EVO 2TB	S6PNNL0T909370N	2TB	22.11.18
b	Samsung SSD 870 EVO 2TB	S6PNNL0T909374Z	2TB	22.11.18
c	Samsung SSD 870 EVO 2TB	S6PNNL0T708760T	2TB	22.11.18
d	Samsung SSD 870 EVO 2TB	S6PNNL0T906382W	2TB	22.11.18

vm5.sea.rg.net (vm1 Replacement)

Dell Poweredge R640 server
Service Tag: CQPCHQ2
2x 10-Core 2.20 GHz Intel CPU's Installed (Silver 4114)
192GB DDR4 (6x32GB) memory installed
PERC H730P Mini RAID controller.
4 x Samsung 4TB EVO 870 SSDs
2x power supplies, 750W each, 100-240V
4x Gigabit RJ45 NIC network ports

vm5 Disks

for i in a b c d; do echo $i; smartctl -a /dev/sd$i | egrep '(^Serial|Model)'; done

Bay	Model	Serial	Capacity	Installed
a	Samsung SSD 870 EVO 4TB	S6PJNS0W307070P	4TB	23.05.21
b	Samsung SSD 870 EVO 4TB	S6PJNS0W307067B	4TB	23.05.21
c	Samsung SSD 870 EVO 4TB	S6PJNS0W307069T	4TB	23.05.21
d	Samsung SSD 870 EVO 4TB	S6PJNS0W307064R	4TB	23.05.21

psg.com

SUPERMICRO AS-1012G-MTF 1U Rackmount H8SGL-F mobo

From for i in 0 1 2 3 4; do smartctl -a /dev/ada$i | egrep 'Model|Serial|Capacity'; done

LUN	Model	Serial	Size	Installed
0	WDC WD2000F9YZ-09N20L0	WD-WCC1P0469969	2TB
1	WDC WD2005FBYZ-01YCBB1	WD-WMC6N0F1F8SD	2TB

LUN	Model	Serial	Size	History
1	WDC WD2005FBYZ-01YCBB1	WD-WMC6N0F1F8SD	2TB	fail 2024.11.17
1	WDC WD2000F9YZ-09N20L0	WD-WCC1P0648270	2TB	removed 2017.04.09
		WD-WCC1P0654846	2TB	replaced 2017.03.04

raid0.sea.rg.net

5018R-WR Chassis + MoBo
X10SRW-F Motherboard
Intel Xeon® Processor E5-2609 v4
Samsung DDR4 2133MHzCL15 16GB RegECC 2Rx4 (PC4 2133) Internal Memory M393A2G40DB0-CPB
FreeBSD USB key left in USB port

raid0.sea.rg.net Drive Bays

From for i in {0..3}; do smartctl -a /dev/ada$i | egrep 'Model|Serial|Capacity'; done

LUN	Model	Serial	Capacity	Date
0	WDC WD60EFRX-68L0BN1	WD-WX41DA5F88N9	6tb	2018.12.09
1	WDC WD6004FRYZ-01CBJB0	AQ01LV7M	6tb	2024.09.26
2	WDC WD60EFRX-68L0BN1	WD-WX11D3678XTL	6tb	2018.12.09
3	WDC WD60EFRX-68L0BN1	WD-WX41D75LN9KC	6tb	2018.12.09

Removed

LUN	Model	Serial	Capacity	Date	Replaced
1	WDC WD60EFRX-68L0BN1	WD-WX11D363T2Y5	6tb	2018.12.09	2024.07.10
1	WDC WD6004FRYZ-01CBJB0	AQ00WHDM	6tb	2024.07.10	2024.09.26

Switches

name	mfr	model	ip sddress
sw0.sea.rg.net	Cisco	WS-C3750G-48PS	147.28.0.253
sw3.sea.rg.net	Cisco	Nexus3064 48x10GE +4	147.28.0.239
sw4.sea.rg.net	Cisco	Nexus3064 48x10GE +4	147.28.0.238

Removed

sw1.sea.rg.net	SMC	SMC8024L2	147.28.0.252
sw2.sea.rg.net	Cisco	WS-C3750G-24PS	147.28.0.246

Miscelaneous Data

Spare Drives


spares in westin

HGST	H3D30003254S	YNG1E3ZC	3TB
Seagate	ST31000528AS	6VPCVN1Y	1TB
Seagate	ST3750640A8	3QD06D5B	750G
HGST	118032816	YFJLBZ5A	2TB

r1.sea gigether pic left to right

port	to	medium
ge-0/0/0	LAN	copper
ge-0/0/1	six	fiber sfp
	unused	copper sfp
	unused

New Switching

Decommmissioned

SIX 1G to r1.sea.rg.net (decomissioned)
 A.OF.12.02
 OCXS2.A06
 MR5A.04.A06
 MR5D.09.C06 (Patch Panel in meet-me-room)

Old Lunavi Info

Lunavi 2001 6th Avenue, Suite 2402 Seattle, WA 98121 Lunavi Support support@lunavi.com 206 777 9988 24x7@ccom.net 206-854-3444 and 206-353-5558

2024.11.29

21 KiB Raw Blame History Unescape Escape

Seattle KOMO PoP

Block Diagram

Administrative Information

KOMO Rack

pwr0 and pwr1 MACs

100mb interfaces on sw0

vm[0-5].sea.rg.net Ganeti Cluster

Guest VMs on vm0/2/3/4/5.sea.rg.net / 147.28.0.14/15/16/29

Back/DRBD Channel

vm0.sea.rg.net Hardware

vm0.sea drives, numbered left to right

vm2.sea.rg.net Hardware

vm2.sea.rg.net Disk Drives

vm3.sea.rg.net Hardware

vm3.sea drives, we guess numbered left to right

vm4.sea.rg.net Hardware

vm5.sea.rg.net (vm1 Replacement)

vm5 Disks

psg.com

raid0.sea.rg.net

raid0.sea.rg.net Drive Bays

Removed

Switches

Removed

Miscelaneous Data

Spare Drives

r1.sea gigether pic left to right

New Switching

Decommmissioned

Old Lunavi Info

21 KiB

Raw Blame History